Note: This is test shard 1 of 3.
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260926 07:35:06.094086 10924 test_util.cc:268] Using random seed: -550851179
2026-09-26T07:35:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:35:06Z Disabled control of system clock
I20260926 07:35:06.149070 10924 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
/tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.171.62:37629
--webserver_interface=127.10.171.62
--webserver_port=0
--builtin_ntp_servers=127.10.171.20:43419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.171.62:37629,127.10.171.61:38823,127.10.171.60:40073
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 07:35:06.447839 10933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:35:06.448480 10933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:35:06.448976 10933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:35:06.478842 10933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:35:06.479198 10933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:35:06.479460 10933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:35:06.479708 10933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:35:06.513442 10933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.171.20:43419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.171.62:37629,127.10.171.61:38823,127.10.171.60:40073
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.171.62:37629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.171.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:55:20 UTC on ccycloud.quasar-qwhbmo.root.comops.site
build id 4673737
TSAN enabled
I20260926 07:35:06.514693 10933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:35:06.516248 10933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:35:06.534842 10942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:35:06.534955 10939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:35:06.535004 10940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:35:06.535967 10933 server_base.cc:1034] running on GCE node
I20260926 07:35:07.713239 10933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:35:07.715911 10933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:35:07.717337 10933 hybrid_clock.cc:648] HybridClock initialized: now 1790408107717310 us; error 50 us; skew 500 ppm
I20260926 07:35:07.718170 10933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:35:07.732517 10933 webserver.cc:460] Webserver started at http://127.10.171.62:37637/ using document root <none> and password file <none>
I20260926 07:35:07.733516 10933 fs_manager.cc:359] Metadata directory not provided
I20260926 07:35:07.733775 10933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:35:07.734256 10933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:35:07.739171 10933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/data/instance:
uuid: "d6ef5b900d0f4e73a358f415efa97249"
format_stamp: "Formatted at 2026-09-26 07:35:07 on dist-test-slave-3mmx"
I20260926 07:35:07.740348 10933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/wal/instance:
uuid: "d6ef5b900d0f4e73a358f415efa97249"
format_stamp: "Formatted at 2026-09-26 07:35:07 on dist-test-slave-3mmx"
I20260926 07:35:07.748391 10933 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260926 07:35:07.753777 10949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:35:07.754752 10933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260926 07:35:07.755076 10933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/data,/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/wal
uuid: "d6ef5b900d0f4e73a358f415efa97249"
format_stamp: "Formatted at 2026-09-26 07:35:07 on dist-test-slave-3mmx"
I20260926 07:35:07.755419 10933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:35:07.803081 10933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:35:07.804534 10933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:35:07.805076 10933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:35:07.877951 10933 rpc_server.cc:307] RPC server started. Bound to: 127.10.171.62:37629
I20260926 07:35:07.878037 11000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.171.62:37629 every 8 connection(s)
I20260926 07:35:07.880645 10933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/data/info.pb
I20260926 07:35:07.885890 11001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:07.887662 10924 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu as pid 10933
I20260926 07:35:07.888206 10924 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/wal/instance
I20260926 07:35:07.893579 10924 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
/tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.171.61:38823
--webserver_interface=127.10.171.61
--webserver_port=0
--builtin_ntp_servers=127.10.171.20:43419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.171.62:37629,127.10.171.61:38823,127.10.171.60:40073
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 07:35:07.901857 11001 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:35:07.930888 11001 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } has no permanent_uuid. Determining permanent_uuid...
W20260926 07:35:07.932907 10950 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.61:38823: connect: Connection refused (error 111)
W20260926 07:35:07.937276 11001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.61:38823: Network error: Client connection negotiation failed: client connection to 127.10.171.61:38823: connect: Connection refused (error 111)
I20260926 07:35:07.986837 11001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } attempt: 1
W20260926 07:35:07.990454 11001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.61:38823: Network error: Client connection negotiation failed: client connection to 127.10.171.61:38823: connect: Connection refused (error 111)
I20260926 07:35:08.058916 11001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } attempt: 2
W20260926 07:35:08.062603 11001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.61:38823: Network error: Client connection negotiation failed: client connection to 127.10.171.61:38823: connect: Connection refused (error 111)
I20260926 07:35:08.154070 11001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } attempt: 3
W20260926 07:35:08.158149 11001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.61:38823: Network error: Client connection negotiation failed: client connection to 127.10.171.61:38823: connect: Connection refused (error 111)
W20260926 07:35:08.203891 11003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:35:08.204862 11003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:35:08.205595 11003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:35:08.238752 11003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:35:08.239111 11003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:35:08.239373 11003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:35:08.239624 11003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:35:08.273909 11003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.171.20:43419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.171.62:37629,127.10.171.61:38823,127.10.171.60:40073
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.171.61:38823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.171.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:55:20 UTC on ccycloud.quasar-qwhbmo.root.comops.site
build id 4673737
TSAN enabled
I20260926 07:35:08.275169 11003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:35:08.276808 11003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:35:08.294490 11015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:35:08.301569 11001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } attempt: 4
W20260926 07:35:08.304979 11001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.61:38823: Network error: Client connection negotiation failed: client connection to 127.10.171.61:38823: connect: Connection refused (error 111)
I20260926 07:35:08.604374 11001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } attempt: 5
W20260926 07:35:08.608636 11001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.61:38823: Network error: Client connection negotiation failed: client connection to 127.10.171.61:38823: connect: Connection refused (error 111)
I20260926 07:35:09.156250 11001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } attempt: 6
W20260926 07:35:09.163213 11001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.61:38823: Network error: Client connection negotiation failed: client connection to 127.10.171.61:38823: connect: Connection refused (error 111)
W20260926 07:35:08.294875 11013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:35:08.294782 11012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:35:09.490056 11014 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260926 07:35:09.490209 11003 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260926 07:35:09.494577 11003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:35:09.497186 11003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:35:09.498559 11003 hybrid_clock.cc:648] HybridClock initialized: now 1790408109498544 us; error 56 us; skew 500 ppm
I20260926 07:35:09.499364 11003 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:35:09.506383 11003 webserver.cc:460] Webserver started at http://127.10.171.61:39027/ using document root <none> and password file <none>
I20260926 07:35:09.507345 11003 fs_manager.cc:359] Metadata directory not provided
I20260926 07:35:09.507592 11003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:35:09.508087 11003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:35:09.513645 11003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-1/data/instance:
uuid: "0773380643194748af6701300513e7a9"
format_stamp: "Formatted at 2026-09-26 07:35:09 on dist-test-slave-3mmx"
I20260926 07:35:09.514776 11003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-1/wal/instance:
uuid: "0773380643194748af6701300513e7a9"
format_stamp: "Formatted at 2026-09-26 07:35:09 on dist-test-slave-3mmx"
I20260926 07:35:09.521831 11003 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260926 07:35:09.527611 11024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:35:09.528625 11003 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 07:35:09.528951 11003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-1/data,/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-1/wal
uuid: "0773380643194748af6701300513e7a9"
format_stamp: "Formatted at 2026-09-26 07:35:09 on dist-test-slave-3mmx"
I20260926 07:35:09.529318 11003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:35:09.560199 11003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:35:09.561614 11003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:35:09.562095 11003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:35:09.635322 11003 rpc_server.cc:307] RPC server started. Bound to: 127.10.171.61:38823
I20260926 07:35:09.635856 11075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.171.61:38823 every 8 connection(s)
I20260926 07:35:09.638052 11003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-1/data/info.pb
I20260926 07:35:09.643208 11076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:09.643616 10924 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu as pid 11003
I20260926 07:35:09.644078 10924 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-1/wal/instance
I20260926 07:35:09.647400 10924 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
/tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.171.60:40073
--webserver_interface=127.10.171.60
--webserver_port=0
--builtin_ntp_servers=127.10.171.20:43419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.171.62:37629,127.10.171.61:38823,127.10.171.60:40073
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 07:35:09.657722 11076 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:35:09.681955 11076 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:35:09.699085 11076 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } has no permanent_uuid. Determining permanent_uuid...
W20260926 07:35:09.701433 11026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.60:40073: connect: Connection refused (error 111)
W20260926 07:35:09.705571 11076 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:40073: Network error: Client connection negotiation failed: client connection to 127.10.171.60:40073: connect: Connection refused (error 111)
I20260926 07:35:09.755062 11076 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } attempt: 1
W20260926 07:35:09.758921 11076 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:40073: Network error: Client connection negotiation failed: client connection to 127.10.171.60:40073: connect: Connection refused (error 111)
I20260926 07:35:09.827312 11076 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } attempt: 2
W20260926 07:35:09.831182 11076 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:40073: Network error: Client connection negotiation failed: client connection to 127.10.171.60:40073: connect: Connection refused (error 111)
I20260926 07:35:09.922593 11076 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } attempt: 3
W20260926 07:35:09.927070 11076 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:40073: Network error: Client connection negotiation failed: client connection to 127.10.171.60:40073: connect: Connection refused (error 111)
W20260926 07:35:09.958416 11078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:35:09.959213 11078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:35:09.959731 11078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:35:09.990346 11078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:35:09.990696 11078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:35:09.990978 11078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:35:09.991241 11078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:35:10.026536 11078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.171.20:43419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.171.62:37629,127.10.171.61:38823,127.10.171.60:40073
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.171.60:40073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.171.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:55:20 UTC on ccycloud.quasar-qwhbmo.root.comops.site
build id 4673737
TSAN enabled
I20260926 07:35:10.027779 11078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:35:10.029325 11078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:35:10.049538 11089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:35:10.070406 11076 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } attempt: 4
W20260926 07:35:10.074683 11076 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:40073: Network error: Client connection negotiation failed: client connection to 127.10.171.60:40073: connect: Connection refused (error 111)
I20260926 07:35:10.223798 11001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } attempt: 7
I20260926 07:35:10.242648 11001 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } has no permanent_uuid. Determining permanent_uuid...
W20260926 07:35:10.248895 11001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:40073: Network error: Client connection negotiation failed: client connection to 127.10.171.60:40073: connect: Connection refused (error 111)
I20260926 07:35:10.307237 11001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } attempt: 1
W20260926 07:35:10.311331 11001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:40073: Network error: Client connection negotiation failed: client connection to 127.10.171.60:40073: connect: Connection refused (error 111)
I20260926 07:35:10.374015 11076 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } attempt: 5
W20260926 07:35:10.378301 11076 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:40073: Network error: Client connection negotiation failed: client connection to 127.10.171.60:40073: connect: Connection refused (error 111)
I20260926 07:35:10.392678 11001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } attempt: 2
W20260926 07:35:10.396698 11001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:40073: Network error: Client connection negotiation failed: client connection to 127.10.171.60:40073: connect: Connection refused (error 111)
I20260926 07:35:10.482070 11001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } attempt: 3
W20260926 07:35:10.486328 11001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:40073: Network error: Client connection negotiation failed: client connection to 127.10.171.60:40073: connect: Connection refused (error 111)
I20260926 07:35:10.626703 11001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } attempt: 4
W20260926 07:35:10.630717 11001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:40073: Network error: Client connection negotiation failed: client connection to 127.10.171.60:40073: connect: Connection refused (error 111)
I20260926 07:35:10.914083 11001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } attempt: 5
W20260926 07:35:10.918849 11001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:40073: Network error: Client connection negotiation failed: client connection to 127.10.171.60:40073: connect: Connection refused (error 111)
I20260926 07:35:10.925884 11076 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } attempt: 6
W20260926 07:35:10.932821 11076 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:40073: Network error: Client connection negotiation failed: client connection to 127.10.171.60:40073: connect: Connection refused (error 111)
W20260926 07:35:10.049549 11088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:35:10.049979 11091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:35:10.052335 11078 server_base.cc:1034] running on GCE node
I20260926 07:35:11.251618 11078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:35:11.254211 11078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:35:11.255573 11078 hybrid_clock.cc:648] HybridClock initialized: now 1790408111255562 us; error 58 us; skew 500 ppm
I20260926 07:35:11.256428 11078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:35:11.262838 11078 webserver.cc:460] Webserver started at http://127.10.171.60:33241/ using document root <none> and password file <none>
I20260926 07:35:11.263748 11078 fs_manager.cc:359] Metadata directory not provided
I20260926 07:35:11.263979 11078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:35:11.264478 11078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:35:11.271077 11078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-2/data/instance:
uuid: "b6e2fc900b374ebab759831b571a208d"
format_stamp: "Formatted at 2026-09-26 07:35:11 on dist-test-slave-3mmx"
I20260926 07:35:11.272254 11078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-2/wal/instance:
uuid: "b6e2fc900b374ebab759831b571a208d"
format_stamp: "Formatted at 2026-09-26 07:35:11 on dist-test-slave-3mmx"
I20260926 07:35:11.279874 11078 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260926 07:35:11.285225 11103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:35:11.286166 11078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.005s
I20260926 07:35:11.286516 11078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-2/data,/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-2/wal
uuid: "b6e2fc900b374ebab759831b571a208d"
format_stamp: "Formatted at 2026-09-26 07:35:11 on dist-test-slave-3mmx"
I20260926 07:35:11.286854 11078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:35:11.326889 11078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:35:11.328420 11078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:35:11.328867 11078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:35:11.403447 11078 rpc_server.cc:307] RPC server started. Bound to: 127.10.171.60:40073
I20260926 07:35:11.403993 11154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.171.60:40073 every 8 connection(s)
I20260926 07:35:11.406222 11078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-2/data/info.pb
I20260926 07:35:11.409384 10924 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu as pid 11078
I20260926 07:35:11.409881 10924 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-2/wal/instance
I20260926 07:35:11.412061 11155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:11.433190 11155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:35:11.462231 11155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:35:11.471261 11001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } attempt: 6
I20260926 07:35:11.479676 11155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:35:11.500025 11001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249: Bootstrap starting.
I20260926 07:35:11.504654 11155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Bootstrap starting.
I20260926 07:35:11.508176 11001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:11.510123 11001 log.cc:824] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249: Log is configured to *not* fsync() on all Append() calls
I20260926 07:35:11.510771 11155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:11.512791 11155 log.cc:824] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Log is configured to *not* fsync() on all Append() calls
I20260926 07:35:11.515467 11001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249: No bootstrap required, opened a new log
I20260926 07:35:11.521006 11155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: No bootstrap required, opened a new log
I20260926 07:35:11.537907 11001 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:11.538638 11001 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:11.539003 11001 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6ef5b900d0f4e73a358f415efa97249, State: Initialized, Role: FOLLOWER
I20260926 07:35:11.540246 11001 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:11.543685 11155 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:11.544330 11155 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:11.544570 11155 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6e2fc900b374ebab759831b571a208d, State: Initialized, Role: FOLLOWER
I20260926 07:35:11.544155 11169 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } } }
I20260926 07:35:11.545014 11169 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:35:11.545344 11155 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:11.545943 11001 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:35:11.548612 11170 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } } }
I20260926 07:35:11.550212 11170 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:35:11.550649 11155 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [sys.catalog]: configured and running, proceeding with master startup.
W20260926 07:35:11.563614 10955 tablet.cc:2369] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:35:11.584040 11109 tablet.cc:2369] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:35:11.587242 11169 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:11.587899 11169 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:11.593181 11130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6ef5b900d0f4e73a358f415efa97249" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6e2fc900b374ebab759831b571a208d" is_pre_election: true
I20260926 07:35:11.594110 11130 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6ef5b900d0f4e73a358f415efa97249 in term 0.
I20260926 07:35:11.597864 10951 leader_election.cc:302] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6e2fc900b374ebab759831b571a208d, d6ef5b900d0f4e73a358f415efa97249; no voters:
I20260926 07:35:11.600832 11193 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 07:35:11.600970 11169 leader_election.cc:288] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0773380643194748af6701300513e7a9 (127.10.171.61:38823), b6e2fc900b374ebab759831b571a208d (127.10.171.60:40073)
W20260926 07:35:11.586762 11189 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260926 07:35:11.603367 11169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:11.606072 11169 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260926 07:35:11.605078 11189 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:35:11.606781 11169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:11.614774 11169 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:11.616827 11169 leader_election.cc:288] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [CANDIDATE]: Term 1 election: Requested vote from peers 0773380643194748af6701300513e7a9 (127.10.171.61:38823), b6e2fc900b374ebab759831b571a208d (127.10.171.60:40073)
I20260926 07:35:11.621600 11130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6ef5b900d0f4e73a358f415efa97249" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6e2fc900b374ebab759831b571a208d"
I20260926 07:35:11.622166 11130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:11.631381 11130 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6ef5b900d0f4e73a358f415efa97249 in term 1.
I20260926 07:35:11.582425 10933 master_runner.cc:186] Error getting master registration for 127.10.171.61:38823: OK, instance_id { permanent_uuid: "0773380643194748af6701300513e7a9" instance_seqno: 1790408109615234 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 07:35:11.634392 10933 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:35:11.636910 10951 leader_election.cc:302] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6e2fc900b374ebab759831b571a208d, d6ef5b900d0f4e73a358f415efa97249; no voters:
I20260926 07:35:11.637589 11169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:11.637944 11169 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [term 1 LEADER]: Becoming Leader. State: Replica: d6ef5b900d0f4e73a358f415efa97249, State: Running, Role: LEADER
I20260926 07:35:11.638603 11169 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:11.660709 11194 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6ef5b900d0f4e73a358f415efa97249. Latest consensus state: current_term: 1 leader_uuid: "d6ef5b900d0f4e73a358f415efa97249" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } } }
I20260926 07:35:11.661513 11194 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [sys.catalog]: This master's current role is: LEADER
I20260926 07:35:11.669811 11196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:35:11.690300 11196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:35:11.637676 11078 master_runner.cc:186] Error getting master registration for 127.10.171.61:38823: OK, instance_id { permanent_uuid: "0773380643194748af6701300513e7a9" instance_seqno: 1790408109615234 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 07:35:11.698251 11078 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:35:11.713047 11130 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Refusing update from remote peer d6ef5b900d0f4e73a358f415efa97249: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:35:11.714563 11169 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:11.745695 11170 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6ef5b900d0f4e73a358f415efa97249. Latest consensus state: current_term: 1 leader_uuid: "d6ef5b900d0f4e73a358f415efa97249" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } } }
I20260926 07:35:11.746719 11170 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:35:11.747660 11197 mvcc.cc:205] Tried to move back new op lower bound from 7333511625545572352 to 7333511625280634880. Current Snapshot: MvccSnapshot[applied={T|T < 7333511625545572352}]
I20260926 07:35:11.749296 11198 mvcc.cc:205] Tried to move back new op lower bound from 7333511625545572352 to 7333511625280634880. Current Snapshot: MvccSnapshot[applied={T|T < 7333511625545572352}]
I20260926 07:35:11.751669 11169 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d6ef5b900d0f4e73a358f415efa97249" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } } }
I20260926 07:35:11.752442 11169 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [sys.catalog]: This master's current role is: LEADER
I20260926 07:35:11.755012 11196 catalog_manager.cc:1347] Generated new cluster ID: d1a1bab5ae3b4961a68dafcdbb937fd6
I20260926 07:35:11.755302 11196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:35:11.756894 11170 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d6ef5b900d0f4e73a358f415efa97249" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } } }
I20260926 07:35:11.757660 11170 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:35:11.787849 11196 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:35:11.790880 11196 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:35:11.834024 11196 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249: Generated new TSK 0
I20260926 07:35:11.835165 11196 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:35:11.856740 10924 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
/tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.171.1:0
--local_ip_for_outbound_sockets=127.10.171.1
--webserver_interface=127.10.171.1
--webserver_port=0
--tserver_master_addrs=127.10.171.62:37629,127.10.171.61:38823,127.10.171.60:40073
--builtin_ntp_servers=127.10.171.20:43419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260926 07:35:11.993360 11076 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } attempt: 7
I20260926 07:35:12.032086 11076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9: Bootstrap starting.
I20260926 07:35:12.039175 11076 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:12.041805 11076 log.cc:824] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9: Log is configured to *not* fsync() on all Append() calls
I20260926 07:35:12.047312 11076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9: No bootstrap required, opened a new log
I20260926 07:35:12.081102 11076 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:12.081965 11076 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:12.082281 11076 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0773380643194748af6701300513e7a9, State: Initialized, Role: FOLLOWER
I20260926 07:35:12.083209 11076 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:12.088929 11076 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:35:12.089841 11051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6ef5b900d0f4e73a358f415efa97249" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0773380643194748af6701300513e7a9" is_pre_election: true
W20260926 07:35:12.090951 11051 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39578 (request call id 2) took 494 ms (client timeout 599 ms). Trace:
I20260926 07:35:12.092135 11050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6ef5b900d0f4e73a358f415efa97249" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0773380643194748af6701300513e7a9"
W20260926 07:35:12.092921 11050 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39578 (request call id 3) took 472 ms (client timeout 602 ms). Trace:
W20260926 07:35:12.093225 11050 rpcz_store.cc:269] 0926 07:35:11.620918 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:11.621062 (+ 144us) service_pool.cc:227] Handling call
0926 07:35:12.092873 (+471811us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260926 07:35:12.089389 11049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 0 FOLLOWER]: Advancing to term 1
W20260926 07:35:12.091272 11051 rpcz_store.cc:269] 0926 07:35:11.596377 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:11.596547 (+ 170us) service_pool.cc:227] Handling call
0926 07:35:12.090886 (+494339us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260926 07:35:12.087204 11205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } } }
I20260926 07:35:12.102275 11205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:35:12.105069 11049 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Refusing update from remote peer d6ef5b900d0f4e73a358f415efa97249: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:35:12.108048 11169 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:12.161008 11205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6ef5b900d0f4e73a358f415efa97249. Latest consensus state: current_term: 1 leader_uuid: "d6ef5b900d0f4e73a358f415efa97249" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } } }
I20260926 07:35:12.161725 11205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:35:12.161175 11210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d6ef5b900d0f4e73a358f415efa97249" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } } }
I20260926 07:35:12.162214 11210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:35:12.199160 11221 catalog_manager.cc:1259] Loaded cluster ID: d1a1bab5ae3b4961a68dafcdbb937fd6
I20260926 07:35:12.199528 11221 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9: loading cluster ID for follower catalog manager: success
I20260926 07:35:12.208599 11221 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9: acquiring CA information for follower catalog manager: success
I20260926 07:35:12.214258 11221 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 07:35:12.327019 11202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:35:12.327724 11202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:35:12.328737 11202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:35:12.385473 11202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:35:12.387112 11202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.171.1
I20260926 07:35:12.445541 11202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.171.20:43419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.171.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.171.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.171.62:37629,127.10.171.61:38823,127.10.171.60:40073
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.171.1
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:55:20 UTC on ccycloud.quasar-qwhbmo.root.comops.site
build id 4673737
TSAN enabled
I20260926 07:35:12.447595 11202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:35:12.449820 11202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:35:12.509557 11228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:35:12.613791 11210 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6ef5b900d0f4e73a358f415efa97249)
I20260926 07:35:12.614560 11210 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:12.618567 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0773380643194748af6701300513e7a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d6ef5b900d0f4e73a358f415efa97249" is_pre_election: true
I20260926 07:35:12.618846 11210 leader_election.cc:288] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), b6e2fc900b374ebab759831b571a208d (127.10.171.60:40073)
I20260926 07:35:12.620460 11130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0773380643194748af6701300513e7a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b6e2fc900b374ebab759831b571a208d" is_pre_election: true
I20260926 07:35:12.622758 11025 leader_election.cc:302] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0773380643194748af6701300513e7a9; no voters: b6e2fc900b374ebab759831b571a208d, d6ef5b900d0f4e73a358f415efa97249
I20260926 07:35:12.623939 11210 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:35:12.627298 11193 catalog_manager.cc:1259] Loaded cluster ID: d1a1bab5ae3b4961a68dafcdbb937fd6
I20260926 07:35:12.627609 11193 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: loading cluster ID for follower catalog manager: success
I20260926 07:35:12.658169 11193 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: acquiring CA information for follower catalog manager: success
I20260926 07:35:12.686231 11193 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 07:35:12.704207 11169 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d6ef5b900d0f4e73a358f415efa97249" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } } }
I20260926 07:35:12.705261 11169 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [sys.catalog]: This master's current role is: LEADER
I20260926 07:35:13.219558 11237 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6ef5b900d0f4e73a358f415efa97249)
I20260926 07:35:13.220247 11237 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:13.222869 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0773380643194748af6701300513e7a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d6ef5b900d0f4e73a358f415efa97249" is_pre_election: true
I20260926 07:35:13.225566 11130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0773380643194748af6701300513e7a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b6e2fc900b374ebab759831b571a208d" is_pre_election: true
I20260926 07:35:13.227033 11026 leader_election.cc:302] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0773380643194748af6701300513e7a9; no voters: b6e2fc900b374ebab759831b571a208d, d6ef5b900d0f4e73a358f415efa97249
I20260926 07:35:13.230255 11237 leader_election.cc:288] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), b6e2fc900b374ebab759831b571a208d (127.10.171.60:40073)
I20260926 07:35:13.231150 11237 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:35:13.281508 11239 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6ef5b900d0f4e73a358f415efa97249)
I20260926 07:35:13.282275 11239 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:13.285650 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e2fc900b374ebab759831b571a208d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d6ef5b900d0f4e73a358f415efa97249" is_pre_election: true
I20260926 07:35:13.288671 11051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e2fc900b374ebab759831b571a208d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0773380643194748af6701300513e7a9" is_pre_election: true
I20260926 07:35:13.290309 11104 leader_election.cc:302] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b6e2fc900b374ebab759831b571a208d; no voters: 0773380643194748af6701300513e7a9, d6ef5b900d0f4e73a358f415efa97249
I20260926 07:35:13.295486 11239 leader_election.cc:288] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), 0773380643194748af6701300513e7a9 (127.10.171.61:38823)
I20260926 07:35:13.297464 11239 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260926 07:35:13.783424 11151 debug-util.cc:398] Leaking SignalData structure 0x7b08000a0280 after lost signal to thread 11083
W20260926 07:35:13.784554 11151 debug-util.cc:398] Leaking SignalData structure 0x7b08000a0620 after lost signal to thread 11154
I20260926 07:35:13.805213 11240 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6ef5b900d0f4e73a358f415efa97249)
I20260926 07:35:13.805685 11240 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:13.807391 11240 leader_election.cc:288] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), 0773380643194748af6701300513e7a9 (127.10.171.61:38823)
I20260926 07:35:13.808827 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e2fc900b374ebab759831b571a208d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d6ef5b900d0f4e73a358f415efa97249" is_pre_election: true
I20260926 07:35:13.810945 11051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e2fc900b374ebab759831b571a208d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0773380643194748af6701300513e7a9" is_pre_election: true
I20260926 07:35:13.812704 11104 leader_election.cc:302] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b6e2fc900b374ebab759831b571a208d; no voters: 0773380643194748af6701300513e7a9, d6ef5b900d0f4e73a358f415efa97249
I20260926 07:35:13.814049 11240 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260926 07:35:12.511965 11231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:35:12.512823 11202 server_base.cc:1034] running on GCE node
W20260926 07:35:12.510365 11229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:35:13.957671 11202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:35:13.967886 11202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:35:13.969368 11202 hybrid_clock.cc:648] HybridClock initialized: now 1790408113969306 us; error 80 us; skew 500 ppm
I20260926 07:35:13.970427 11202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:35:14.020879 11202 webserver.cc:460] Webserver started at http://127.10.171.1:32817/ using document root <none> and password file <none>
I20260926 07:35:14.022593 11202 fs_manager.cc:359] Metadata directory not provided
I20260926 07:35:14.022977 11202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:35:14.023653 11202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:35:14.042055 11202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/data/instance:
uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
format_stamp: "Formatted at 2026-09-26 07:35:14 on dist-test-slave-3mmx"
I20260926 07:35:14.043766 11202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/instance:
uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
format_stamp: "Formatted at 2026-09-26 07:35:14 on dist-test-slave-3mmx"
I20260926 07:35:14.064949 11202 fs_manager.cc:691] Time spent creating directory manager: real 0.020s user 0.004s sys 0.004s
I20260926 07:35:14.083968 11245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:35:14.097314 11202 fs_manager.cc:725] Time spent opening block manager: real 0.028s user 0.004s sys 0.002s
I20260926 07:35:14.097868 11202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal
uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
format_stamp: "Formatted at 2026-09-26 07:35:14 on dist-test-slave-3mmx"
I20260926 07:35:14.098433 11202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:35:14.195705 11202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:35:14.203173 11202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:35:14.204226 11202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:35:14.218140 11202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:35:14.230367 11251 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6ef5b900d0f4e73a358f415efa97249)
I20260926 07:35:14.236040 11202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:35:14.236347 11202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260926 07:35:14.236653 11202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:35:14.236363 11251 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:14.236855 11202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:35:14.238345 11251 leader_election.cc:288] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), b6e2fc900b374ebab759831b571a208d (127.10.171.60:40073)
I20260926 07:35:14.242107 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0773380643194748af6701300513e7a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d6ef5b900d0f4e73a358f415efa97249" is_pre_election: true
I20260926 07:35:14.243209 11130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0773380643194748af6701300513e7a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b6e2fc900b374ebab759831b571a208d" is_pre_election: true
I20260926 07:35:14.245875 11025 leader_election.cc:302] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0773380643194748af6701300513e7a9; no voters: b6e2fc900b374ebab759831b571a208d, d6ef5b900d0f4e73a358f415efa97249
I20260926 07:35:14.246863 11251 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:35:14.350361 11264 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6ef5b900d0f4e73a358f415efa97249)
I20260926 07:35:14.351666 11264 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:14.360373 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e2fc900b374ebab759831b571a208d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d6ef5b900d0f4e73a358f415efa97249" is_pre_election: true
I20260926 07:35:14.364581 11051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e2fc900b374ebab759831b571a208d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0773380643194748af6701300513e7a9" is_pre_election: true
I20260926 07:35:14.365984 11104 leader_election.cc:302] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b6e2fc900b374ebab759831b571a208d; no voters: 0773380643194748af6701300513e7a9, d6ef5b900d0f4e73a358f415efa97249
I20260926 07:35:14.372718 11264 leader_election.cc:288] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), 0773380643194748af6701300513e7a9 (127.10.171.61:38823)
I20260926 07:35:14.373488 11264 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:35:14.786755 11326 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6ef5b900d0f4e73a358f415efa97249)
I20260926 07:35:14.787431 11326 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:14.792439 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0773380643194748af6701300513e7a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d6ef5b900d0f4e73a358f415efa97249" is_pre_election: true
I20260926 07:35:14.795205 11130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0773380643194748af6701300513e7a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b6e2fc900b374ebab759831b571a208d" is_pre_election: true
I20260926 07:35:14.796705 11026 leader_election.cc:302] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0773380643194748af6701300513e7a9; no voters: b6e2fc900b374ebab759831b571a208d, d6ef5b900d0f4e73a358f415efa97249
I20260926 07:35:14.806617 11326 leader_election.cc:288] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), b6e2fc900b374ebab759831b571a208d (127.10.171.60:40073)
I20260926 07:35:14.809846 11326 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:35:15.026480 11362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.171.1:41829 every 8 connection(s)
I20260926 07:35:15.027001 11202 rpc_server.cc:307] RPC server started. Bound to: 127.10.171.1:41829
I20260926 07:35:15.030577 11202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/data/info.pb
I20260926 07:35:15.033128 10924 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu as pid 11202
I20260926 07:35:15.033584 10924 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/instance
I20260926 07:35:15.054203 10924 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
/tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.171.2:0
--local_ip_for_outbound_sockets=127.10.171.2
--webserver_interface=127.10.171.2
--webserver_port=0
--tserver_master_addrs=127.10.171.62:37629,127.10.171.61:38823,127.10.171.60:40073
--builtin_ntp_servers=127.10.171.20:43419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260926 07:35:15.102999 11363 heartbeater.cc:347] Connected to a master server at 127.10.171.60:40073
I20260926 07:35:15.103478 11363 heartbeater.cc:464] Registering TS with master...
I20260926 07:35:15.104763 11363 heartbeater.cc:511] Master 127.10.171.60:40073 requested a full tablet report, sending...
I20260926 07:35:15.110952 11120 ts_manager.cc:194] Registered new tserver with Master: e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:15.131197 11366 heartbeater.cc:347] Connected to a master server at 127.10.171.61:38823
I20260926 07:35:15.131546 11366 heartbeater.cc:464] Registering TS with master...
I20260926 07:35:15.132340 11366 heartbeater.cc:511] Master 127.10.171.61:38823 requested a full tablet report, sending...
I20260926 07:35:15.135218 11041 ts_manager.cc:194] Registered new tserver with Master: e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:15.136090 11370 heartbeater.cc:347] Connected to a master server at 127.10.171.62:37629
I20260926 07:35:15.136416 11370 heartbeater.cc:464] Registering TS with master...
I20260926 07:35:15.137292 11370 heartbeater.cc:511] Master 127.10.171.62:37629 requested a full tablet report, sending...
I20260926 07:35:15.139657 10966 ts_manager.cc:194] Registered new tserver with Master: e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:15.142984 10966 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.171.1:55773
I20260926 07:35:15.326807 11374 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6ef5b900d0f4e73a358f415efa97249)
I20260926 07:35:15.329568 11374 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:15.336614 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e2fc900b374ebab759831b571a208d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d6ef5b900d0f4e73a358f415efa97249" is_pre_election: true
I20260926 07:35:15.339371 11051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e2fc900b374ebab759831b571a208d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0773380643194748af6701300513e7a9" is_pre_election: true
I20260926 07:35:15.340827 11104 leader_election.cc:302] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b6e2fc900b374ebab759831b571a208d; no voters: 0773380643194748af6701300513e7a9, d6ef5b900d0f4e73a358f415efa97249
I20260926 07:35:15.346098 11374 leader_election.cc:288] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), 0773380643194748af6701300513e7a9 (127.10.171.61:38823)
I20260926 07:35:15.348137 11374 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:35:15.647188 11370 heartbeater.cc:503] Master 127.10.171.62:37629 was elected leader, sending a full tablet report...
W20260926 07:35:15.686978 11369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:35:15.694228 11369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:35:15.695254 11369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:35:15.778175 11369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:35:15.779763 11369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.171.2
I20260926 07:35:15.804751 11380 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6ef5b900d0f4e73a358f415efa97249)
I20260926 07:35:15.805338 11380 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:15.808001 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0773380643194748af6701300513e7a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d6ef5b900d0f4e73a358f415efa97249" is_pre_election: true
I20260926 07:35:15.813638 11130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0773380643194748af6701300513e7a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b6e2fc900b374ebab759831b571a208d" is_pre_election: true
I20260926 07:35:15.815194 11026 leader_election.cc:302] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0773380643194748af6701300513e7a9; no voters: b6e2fc900b374ebab759831b571a208d, d6ef5b900d0f4e73a358f415efa97249
I20260926 07:35:15.820677 11380 leader_election.cc:288] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), b6e2fc900b374ebab759831b571a208d (127.10.171.60:40073)
I20260926 07:35:15.821566 11380 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:35:15.854118 11369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.171.20:43419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.171.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.171.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.171.62:37629,127.10.171.61:38823,127.10.171.60:40073
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.171.2
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:55:20 UTC on ccycloud.quasar-qwhbmo.root.comops.site
build id 4673737
TSAN enabled
I20260926 07:35:15.855515 11369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:35:15.858042 11369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:35:15.908859 11383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:35:15.917086 11385 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6ef5b900d0f4e73a358f415efa97249)
I20260926 07:35:15.919067 11385 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:15.925133 11385 leader_election.cc:288] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), 0773380643194748af6701300513e7a9 (127.10.171.61:38823)
I20260926 07:35:15.926493 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e2fc900b374ebab759831b571a208d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d6ef5b900d0f4e73a358f415efa97249" is_pre_election: true
I20260926 07:35:15.927704 11051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e2fc900b374ebab759831b571a208d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0773380643194748af6701300513e7a9" is_pre_election: true
I20260926 07:35:15.931164 11104 leader_election.cc:302] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b6e2fc900b374ebab759831b571a208d; no voters: 0773380643194748af6701300513e7a9, d6ef5b900d0f4e73a358f415efa97249
I20260926 07:35:15.931915 11385 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:35:16.907796 11390 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6ef5b900d0f4e73a358f415efa97249)
I20260926 07:35:16.909936 11390 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:16.913847 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0773380643194748af6701300513e7a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d6ef5b900d0f4e73a358f415efa97249" is_pre_election: true
I20260926 07:35:16.921083 11130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0773380643194748af6701300513e7a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b6e2fc900b374ebab759831b571a208d" is_pre_election: true
I20260926 07:35:16.923733 11026 leader_election.cc:302] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0773380643194748af6701300513e7a9; no voters: b6e2fc900b374ebab759831b571a208d, d6ef5b900d0f4e73a358f415efa97249
I20260926 07:35:16.928850 11390 leader_election.cc:288] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), b6e2fc900b374ebab759831b571a208d (127.10.171.60:40073)
I20260926 07:35:16.932626 11390 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260926 07:35:15.910193 11384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:35:17.278141 11382 debug-util.cc:398] Leaking SignalData structure 0x7b08000148c0 after lost signal to thread 11369
W20260926 07:35:17.438656 11369 thread.cc:636] GCE (cloud detector) Time spent creating pthread: real 1.535s user 0.514s sys 1.003s
W20260926 07:35:17.439078 11369 thread.cc:602] GCE (cloud detector) Time spent starting thread: real 1.536s user 0.515s sys 1.004s
I20260926 07:35:17.445930 11369 server_base.cc:1034] running on GCE node
W20260926 07:35:17.446008 11392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:35:17.447288 11369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:35:17.449352 11369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:35:17.450708 11369 hybrid_clock.cc:648] HybridClock initialized: now 1790408117450676 us; error 38 us; skew 500 ppm
I20260926 07:35:17.451742 11369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:35:17.458721 11369 webserver.cc:460] Webserver started at http://127.10.171.2:34825/ using document root <none> and password file <none>
I20260926 07:35:17.459694 11369 fs_manager.cc:359] Metadata directory not provided
I20260926 07:35:17.459874 11369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:35:17.460307 11369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:35:17.465157 11369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/data/instance:
uuid: "22f3902d01584936b20437606cb86dd1"
format_stamp: "Formatted at 2026-09-26 07:35:17 on dist-test-slave-3mmx"
I20260926 07:35:17.466218 11369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/instance:
uuid: "22f3902d01584936b20437606cb86dd1"
format_stamp: "Formatted at 2026-09-26 07:35:17 on dist-test-slave-3mmx"
I20260926 07:35:17.473119 11369 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260926 07:35:17.478657 11398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:35:17.479621 11369 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.001s
I20260926 07:35:17.479877 11369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal
uuid: "22f3902d01584936b20437606cb86dd1"
format_stamp: "Formatted at 2026-09-26 07:35:17 on dist-test-slave-3mmx"
I20260926 07:35:17.480227 11369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:35:17.523343 11369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:35:17.524778 11369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:35:17.525161 11369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:35:17.527467 11369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:35:17.531239 11369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:35:17.531416 11369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:35:17.531620 11369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:35:17.531764 11369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:35:17.681098 11369 rpc_server.cc:307] RPC server started. Bound to: 127.10.171.2:42379
I20260926 07:35:17.681164 11510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.171.2:42379 every 8 connection(s)
I20260926 07:35:17.684516 11369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/data/info.pb
I20260926 07:35:17.688166 10924 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu as pid 11369
I20260926 07:35:17.689205 10924 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/instance
I20260926 07:35:17.704336 10924 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
/tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.171.3:0
--local_ip_for_outbound_sockets=127.10.171.3
--webserver_interface=127.10.171.3
--webserver_port=0
--tserver_master_addrs=127.10.171.62:37629,127.10.171.61:38823,127.10.171.60:40073
--builtin_ntp_servers=127.10.171.20:43419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260926 07:35:17.735728 11512 heartbeater.cc:347] Connected to a master server at 127.10.171.61:38823
I20260926 07:35:17.736240 11512 heartbeater.cc:464] Registering TS with master...
I20260926 07:35:17.737394 11512 heartbeater.cc:511] Master 127.10.171.61:38823 requested a full tablet report, sending...
I20260926 07:35:17.737884 11511 heartbeater.cc:347] Connected to a master server at 127.10.171.60:40073
I20260926 07:35:17.738217 11511 heartbeater.cc:464] Registering TS with master...
I20260926 07:35:17.738922 11511 heartbeater.cc:511] Master 127.10.171.60:40073 requested a full tablet report, sending...
I20260926 07:35:17.739763 11515 heartbeater.cc:347] Connected to a master server at 127.10.171.62:37629
I20260926 07:35:17.740067 11515 heartbeater.cc:464] Registering TS with master...
I20260926 07:35:17.740818 11515 heartbeater.cc:511] Master 127.10.171.62:37629 requested a full tablet report, sending...
I20260926 07:35:17.741223 11040 ts_manager.cc:194] Registered new tserver with Master: 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:17.741251 11120 ts_manager.cc:194] Registered new tserver with Master: 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:17.742621 10966 ts_manager.cc:194] Registered new tserver with Master: 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:17.744366 10966 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.171.2:34493
I20260926 07:35:17.909060 11522 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6ef5b900d0f4e73a358f415efa97249)
I20260926 07:35:17.909471 11522 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:17.911307 11522 leader_election.cc:288] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), b6e2fc900b374ebab759831b571a208d (127.10.171.60:40073)
I20260926 07:35:17.912428 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0773380643194748af6701300513e7a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d6ef5b900d0f4e73a358f415efa97249" is_pre_election: true
I20260926 07:35:17.912751 11130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0773380643194748af6701300513e7a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b6e2fc900b374ebab759831b571a208d" is_pre_election: true
I20260926 07:35:17.913904 11026 leader_election.cc:302] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0773380643194748af6701300513e7a9; no voters: b6e2fc900b374ebab759831b571a208d, d6ef5b900d0f4e73a358f415efa97249
I20260926 07:35:17.914469 11522 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260926 07:35:18.039883 11521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:35:18.040460 11521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:35:18.041113 11521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:35:18.073805 11521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:35:18.074736 11521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.171.3
I20260926 07:35:18.109738 11521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.171.20:43419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.171.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.171.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.171.62:37629,127.10.171.61:38823,127.10.171.60:40073
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.171.3
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:55:20 UTC on ccycloud.quasar-qwhbmo.root.comops.site
build id 4673737
TSAN enabled
I20260926 07:35:18.110854 11521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:35:18.112291 11521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:35:18.132774 11529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:35:18.248201 11515 heartbeater.cc:503] Master 127.10.171.62:37629 was elected leader, sending a full tablet report...
I20260926 07:35:18.351615 11535 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6ef5b900d0f4e73a358f415efa97249)
I20260926 07:35:18.352144 11535 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:18.353847 11535 leader_election.cc:288] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), 0773380643194748af6701300513e7a9 (127.10.171.61:38823)
I20260926 07:35:18.355113 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e2fc900b374ebab759831b571a208d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d6ef5b900d0f4e73a358f415efa97249" is_pre_election: true
I20260926 07:35:18.356133 11051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e2fc900b374ebab759831b571a208d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0773380643194748af6701300513e7a9" is_pre_election: true
I20260926 07:35:18.358418 11104 leader_election.cc:302] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b6e2fc900b374ebab759831b571a208d; no voters: 0773380643194748af6701300513e7a9, d6ef5b900d0f4e73a358f415efa97249
I20260926 07:35:18.359030 11535 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:35:18.875814 11536 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6ef5b900d0f4e73a358f415efa97249)
I20260926 07:35:18.876365 11536 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:18.878801 11536 leader_election.cc:288] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), b6e2fc900b374ebab759831b571a208d (127.10.171.60:40073)
I20260926 07:35:18.879881 11130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0773380643194748af6701300513e7a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b6e2fc900b374ebab759831b571a208d" is_pre_election: true
I20260926 07:35:18.879715 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0773380643194748af6701300513e7a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d6ef5b900d0f4e73a358f415efa97249" is_pre_election: true
I20260926 07:35:18.881739 11025 leader_election.cc:302] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0773380643194748af6701300513e7a9; no voters: b6e2fc900b374ebab759831b571a208d, d6ef5b900d0f4e73a358f415efa97249
I20260926 07:35:18.882484 11536 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:35:18.920576 11537 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6ef5b900d0f4e73a358f415efa97249)
I20260926 07:35:18.921190 11537 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:18.923319 11537 leader_election.cc:288] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), 0773380643194748af6701300513e7a9 (127.10.171.61:38823)
I20260926 07:35:18.924355 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e2fc900b374ebab759831b571a208d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d6ef5b900d0f4e73a358f415efa97249" is_pre_election: true
I20260926 07:35:18.925295 11051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e2fc900b374ebab759831b571a208d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0773380643194748af6701300513e7a9" is_pre_election: true
I20260926 07:35:18.926972 11104 leader_election.cc:302] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b6e2fc900b374ebab759831b571a208d; no voters: 0773380643194748af6701300513e7a9, d6ef5b900d0f4e73a358f415efa97249
I20260926 07:35:18.927529 11537 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:35:18.135387 11521 server_base.cc:1034] running on GCE node
W20260926 07:35:18.133682 11528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:35:18.132782 11531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:35:19.337726 11521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:35:19.340080 11521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:35:19.341465 11521 hybrid_clock.cc:648] HybridClock initialized: now 1790408119341436 us; error 42 us; skew 500 ppm
I20260926 07:35:19.342274 11521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:35:19.353974 11521 webserver.cc:460] Webserver started at http://127.10.171.3:45197/ using document root <none> and password file <none>
I20260926 07:35:19.354890 11521 fs_manager.cc:359] Metadata directory not provided
I20260926 07:35:19.355118 11521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:35:19.355567 11521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:35:19.360414 11521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/data/instance:
uuid: "6070e9e3d1234a9d83e49431416afade"
format_stamp: "Formatted at 2026-09-26 07:35:19 on dist-test-slave-3mmx"
I20260926 07:35:19.361461 11521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/instance:
uuid: "6070e9e3d1234a9d83e49431416afade"
format_stamp: "Formatted at 2026-09-26 07:35:19 on dist-test-slave-3mmx"
I20260926 07:35:19.368434 11521 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 07:35:19.373615 11543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:35:19.374729 11521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260926 07:35:19.375056 11521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal
uuid: "6070e9e3d1234a9d83e49431416afade"
format_stamp: "Formatted at 2026-09-26 07:35:19 on dist-test-slave-3mmx"
I20260926 07:35:19.375429 11521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:35:19.428303 11521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:35:19.429668 11521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:35:19.430145 11521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:35:19.432655 11521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:35:19.436607 11521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:35:19.436793 11521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:35:19.437103 11521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:35:19.437265 11521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:35:19.582548 11521 rpc_server.cc:307] RPC server started. Bound to: 127.10.171.3:45127
I20260926 07:35:19.582643 11655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.171.3:45127 every 8 connection(s)
I20260926 07:35:19.585139 11521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/data/info.pb
I20260926 07:35:19.587288 10924 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu as pid 11521
I20260926 07:35:19.587714 10924 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/instance
I20260926 07:35:19.637633 11656 heartbeater.cc:347] Connected to a master server at 127.10.171.60:40073
I20260926 07:35:19.638090 11656 heartbeater.cc:464] Registering TS with master...
I20260926 07:35:19.639118 11656 heartbeater.cc:511] Master 127.10.171.60:40073 requested a full tablet report, sending...
I20260926 07:35:19.640522 11660 heartbeater.cc:347] Connected to a master server at 127.10.171.62:37629
I20260926 07:35:19.640838 11660 heartbeater.cc:464] Registering TS with master...
I20260926 07:35:19.641774 11660 heartbeater.cc:511] Master 127.10.171.62:37629 requested a full tablet report, sending...
I20260926 07:35:19.644497 11120 ts_manager.cc:194] Registered new tserver with Master: 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:19.644577 10966 ts_manager.cc:194] Registered new tserver with Master: 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:19.646494 10966 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.171.3:49131
I20260926 07:35:19.651064 11657 heartbeater.cc:347] Connected to a master server at 127.10.171.61:38823
I20260926 07:35:19.651438 11657 heartbeater.cc:464] Registering TS with master...
I20260926 07:35:19.652321 11657 heartbeater.cc:511] Master 127.10.171.61:38823 requested a full tablet report, sending...
I20260926 07:35:19.655568 11040 ts_manager.cc:194] Registered new tserver with Master: 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:19.673622 10924 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:35:19.732918 10966 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43136:
name: "default.Table_8bcc28ac71e84597963005d3ef31345b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:19.733232 10965 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43136:
name: "default.Table_ba67df50a2cd47f7bb409d73211d1db0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:19.735317 10964 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43136:
name: "default.Table_02fe86b869794353adca0276c7cd9410"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:19.736986 10964 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_02fe86b869794353adca0276c7cd9410 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:19.737483 10963 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43136:
name: "default.Table_dd596536aeae4424879b483396f10f5a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:19.737807 10966 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8bcc28ac71e84597963005d3ef31345b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:35:19.740211 10963 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dd596536aeae4424879b483396f10f5a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:35:19.740201 10965 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba67df50a2cd47f7bb409d73211d1db0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:19.805274 10924 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu with pid 10933
I20260926 07:35:20.154791 11660 heartbeater.cc:503] Master 127.10.171.62:37629 was elected leader, sending a full tablet report...
W20260926 07:35:20.159389 11660 heartbeater.cc:644] Failed to heartbeat to 127.10.171.62:37629 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.171.62:37629: connect: Connection refused (error 111)
W20260926 07:35:20.185530 11370 heartbeater.cc:644] Failed to heartbeat to 127.10.171.62:37629 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.171.62:37629: connect: Connection refused (error 111)
W20260926 07:35:20.269227 11515 heartbeater.cc:644] Failed to heartbeat to 127.10.171.62:37629 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.171.62:37629: connect: Connection refused (error 111)
I20260926 07:35:20.273069 11701 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6ef5b900d0f4e73a358f415efa97249)
I20260926 07:35:20.273571 11701 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:20.275089 11702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d6ef5b900d0f4e73a358f415efa97249)
I20260926 07:35:20.275410 11701 leader_election.cc:288] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), 0773380643194748af6701300513e7a9 (127.10.171.61:38823)
I20260926 07:35:20.275576 11702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:20.278595 11702 leader_election.cc:288] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), b6e2fc900b374ebab759831b571a208d (127.10.171.60:40073)
I20260926 07:35:20.278421 11051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e2fc900b374ebab759831b571a208d" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "0773380643194748af6701300513e7a9" is_pre_election: true
W20260926 07:35:20.279574 11104 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:37629: connect: Connection refused (error 111)
I20260926 07:35:20.279759 11051 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6e2fc900b374ebab759831b571a208d in term 1.
I20260926 07:35:20.280504 11130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0773380643194748af6701300513e7a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "b6e2fc900b374ebab759831b571a208d" is_pre_election: true
I20260926 07:35:20.281383 11130 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0773380643194748af6701300513e7a9 in term 1.
I20260926 07:35:20.282429 11104 leader_election.cc:302] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0773380643194748af6701300513e7a9, b6e2fc900b374ebab759831b571a208d; no voters:
I20260926 07:35:20.283005 11026 leader_election.cc:302] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0773380643194748af6701300513e7a9, b6e2fc900b374ebab759831b571a208d; no voters:
I20260926 07:35:20.283195 11701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 07:35:20.283471 11701 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Starting leader election (detected failure of leader d6ef5b900d0f4e73a358f415efa97249)
I20260926 07:35:20.283813 11701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 1 FOLLOWER]: Advancing to term 2
I20260926 07:35:20.287859 11702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 07:35:20.288630 11702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d6ef5b900d0f4e73a358f415efa97249)
I20260926 07:35:20.289060 11702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 1 FOLLOWER]: Advancing to term 2
W20260926 07:35:20.290405 11025 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:37629: connect: Connection refused (error 111) [suppressed 6 similar messages]
W20260926 07:35:20.293829 11104 leader_election.cc:334] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629): Network error: Client connection negotiation failed: client connection to 127.10.171.62:37629: connect: Connection refused (error 111)
W20260926 07:35:20.295677 11025 leader_election.cc:334] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629): Network error: Client connection negotiation failed: client connection to 127.10.171.62:37629: connect: Connection refused (error 111)
I20260926 07:35:20.298022 11701 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:20.300625 11702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:20.300927 11701 leader_election.cc:288] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 2 election: Requested vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), 0773380643194748af6701300513e7a9 (127.10.171.61:38823)
I20260926 07:35:20.304270 11051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e2fc900b374ebab759831b571a208d" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "0773380643194748af6701300513e7a9"
I20260926 07:35:20.305367 11051 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b6e2fc900b374ebab759831b571a208d in current term 2: Already voted for candidate 0773380643194748af6701300513e7a9 in this term.
W20260926 07:35:20.306195 11025 leader_election.cc:334] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629): Network error: Client connection negotiation failed: client connection to 127.10.171.62:37629: connect: Connection refused (error 111)
I20260926 07:35:20.308081 11130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0773380643194748af6701300513e7a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "b6e2fc900b374ebab759831b571a208d"
I20260926 07:35:20.308831 11130 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0773380643194748af6701300513e7a9 in current term 2: Already voted for candidate b6e2fc900b374ebab759831b571a208d in this term.
I20260926 07:35:20.309697 11702 leader_election.cc:288] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 election: Requested vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), b6e2fc900b374ebab759831b571a208d (127.10.171.60:40073)
W20260926 07:35:20.310036 11104 leader_election.cc:334] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629): Network error: Client connection negotiation failed: client connection to 127.10.171.62:37629: connect: Connection refused (error 111)
I20260926 07:35:20.309930 11026 leader_election.cc:302] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0773380643194748af6701300513e7a9; no voters: b6e2fc900b374ebab759831b571a208d, d6ef5b900d0f4e73a358f415efa97249
I20260926 07:35:20.310458 11104 leader_election.cc:302] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b6e2fc900b374ebab759831b571a208d; no voters: 0773380643194748af6701300513e7a9, d6ef5b900d0f4e73a358f415efa97249
I20260926 07:35:20.311396 11701 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260926 07:35:20.311600 11702 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260926 07:35:20.907280 11710 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:20.907719 11710 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:20.909159 11710 leader_election.cc:288] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), 0773380643194748af6701300513e7a9 (127.10.171.61:38823)
I20260926 07:35:20.911199 11051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e2fc900b374ebab759831b571a208d" candidate_term: 3 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "0773380643194748af6701300513e7a9" is_pre_election: true
I20260926 07:35:20.911711 11051 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6e2fc900b374ebab759831b571a208d in term 2.
I20260926 07:35:20.912760 11104 leader_election.cc:302] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0773380643194748af6701300513e7a9, b6e2fc900b374ebab759831b571a208d; no voters:
I20260926 07:35:20.913369 11710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260926 07:35:20.913713 11710 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:20.914024 11710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 2 FOLLOWER]: Advancing to term 3
W20260926 07:35:20.914601 11104 leader_election.cc:334] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629): Network error: Client connection negotiation failed: client connection to 127.10.171.62:37629: connect: Connection refused (error 111)
I20260926 07:35:20.918952 11710 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:20.920495 11710 leader_election.cc:288] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 3 election: Requested vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), 0773380643194748af6701300513e7a9 (127.10.171.61:38823)
I20260926 07:35:20.921862 11051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e2fc900b374ebab759831b571a208d" candidate_term: 3 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "0773380643194748af6701300513e7a9"
I20260926 07:35:20.922434 11051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 2 FOLLOWER]: Advancing to term 3
W20260926 07:35:20.923185 11104 leader_election.cc:334] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629): Network error: Client connection negotiation failed: client connection to 127.10.171.62:37629: connect: Connection refused (error 111)
I20260926 07:35:20.929510 11051 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6e2fc900b374ebab759831b571a208d in term 3.
I20260926 07:35:20.930814 11104 leader_election.cc:302] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0773380643194748af6701300513e7a9, b6e2fc900b374ebab759831b571a208d; no voters: d6ef5b900d0f4e73a358f415efa97249
I20260926 07:35:20.931466 11710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 3 FOLLOWER]: Leader election won for term 3
I20260926 07:35:20.932580 11710 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [term 3 LEADER]: Becoming Leader. State: Replica: b6e2fc900b374ebab759831b571a208d, State: Running, Role: LEADER
I20260926 07:35:20.933367 11710 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:20.941618 11713 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6e2fc900b374ebab759831b571a208d. Latest consensus state: current_term: 3 leader_uuid: "b6e2fc900b374ebab759831b571a208d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } } }
I20260926 07:35:20.942414 11713 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [sys.catalog]: This master's current role is: LEADER
W20260926 07:35:21.009056 10929 debug-util.cc:398] Leaking SignalData structure 0x7b080002a0a0 after lost signal to thread 11666
W20260926 07:35:21.136279 10929 kernel_stack_watchdog.cc:198] Thread 11666 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/outbound_call.cc:308 for 178ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260926 07:35:19.831625 11666 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.171.62:37629)
W20260926 07:35:21.138617 11666 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.CreateTable -> {remote=127.10.171.62:37629, user_credentials={real_user=slave}} blocked reactor thread for 1.30902e+06us
I20260926 07:35:21.138921 10924 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
/tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.171.62:37629
--webserver_interface=127.10.171.62
--webserver_port=37637
--builtin_ntp_servers=127.10.171.20:43419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.171.62:37629,127.10.171.61:38823,127.10.171.60:40073
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 07:35:21.145598 11666 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.171.62:37629)
W20260926 07:35:21.409237 11104 consensus_peers.cc:487] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d -> Peer d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629): Couldn't send request to peer d6ef5b900d0f4e73a358f415efa97249. Status: Network error: Client connection negotiation failed: client connection to 127.10.171.62:37629: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:21.441792 11051 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 3 FOLLOWER]: Refusing update from remote peer b6e2fc900b374ebab759831b571a208d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260926 07:35:21.443008 11713 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260926 07:35:21.449430 11721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6e2fc900b374ebab759831b571a208d. Latest consensus state: current_term: 3 leader_uuid: "b6e2fc900b374ebab759831b571a208d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } } }
I20260926 07:35:21.450332 11721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:35:21.459549 11727 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "b6e2fc900b374ebab759831b571a208d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } } }
I20260926 07:35:21.460359 11727 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [sys.catalog]: This master's current role is: LEADER
I20260926 07:35:21.461944 11715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:35:21.466842 11721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "b6e2fc900b374ebab759831b571a208d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } } }
I20260926 07:35:21.467782 11721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:35:21.472633 11715 catalog_manager.cc:669] Loaded metadata for table default.Table_dd596536aeae4424879b483396f10f5a [id=6a3b8de693cc4463835fb6e6aba8cad3]
I20260926 07:35:21.478580 11715 tablet_loader.cc:96] loaded metadata for tablet 4ffd8bef90f14047b367a05295e976dd (table default.Table_dd596536aeae4424879b483396f10f5a [id=6a3b8de693cc4463835fb6e6aba8cad3])
I20260926 07:35:21.479950 11715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:35:21.483706 11715 catalog_manager.cc:1259] Loaded cluster ID: d1a1bab5ae3b4961a68dafcdbb937fd6
I20260926 07:35:21.483934 11715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:35:21.488602 11715 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:35:21.492386 11715 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Loaded TSK: 0
I20260926 07:35:21.493818 11715 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 07:35:21.504290 11716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:35:21.504897 11716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:35:21.505366 11716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 07:35:21.532680 11119 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_02fe86b869794353adca0276c7cd9410"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:21.532377 11120 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_dd596536aeae4424879b483396f10f5a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:21.533663 11117 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_8bcc28ac71e84597963005d3ef31345b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:21.535285 11118 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_ba67df50a2cd47f7bb409d73211d1db0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:21.536270 11117 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8bcc28ac71e84597963005d3ef31345b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:35:21.536285 11119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_02fe86b869794353adca0276c7cd9410 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:35:21.537011 11118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba67df50a2cd47f7bb409d73211d1db0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:35:21.537617 11120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dd596536aeae4424879b483396f10f5a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:21.542945 11120 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_f9373600df8e4f87bf996753d4f7c975"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:21.544391 11120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f9373600df8e4f87bf996753d4f7c975 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:35:21.546137 11716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:35:21.546540 11716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:35:21.546887 11716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:35:21.547205 11716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:35:21.573261 11118 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_02fe86b869794353adca0276c7cd9410" } new_table_name: "default.Table_9d17125d78a24e0e856f15ae94bf3efa" modify_external_catalogs: true
I20260926 07:35:21.577996 11117 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_8bcc28ac71e84597963005d3ef31345b" } new_table_name: "default.Table_96ce23d5255e41f8a900ac36b2ebfa54" modify_external_catalogs: true
I20260926 07:35:21.579442 11119 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_ba67df50a2cd47f7bb409d73211d1db0" } modify_external_catalogs: true
I20260926 07:35:21.580071 11119 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_ba67df50a2cd47f7bb409d73211d1db0" } modify_external_catalogs: true
I20260926 07:35:21.594887 11120 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_f9373600df8e4f87bf996753d4f7c975" } modify_external_catalogs: true
I20260926 07:35:21.595449 11120 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_f9373600df8e4f87bf996753d4f7c975" } modify_external_catalogs: true
I20260926 07:35:21.618755 11716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.171.20:43419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.171.62:37629,127.10.171.61:38823,127.10.171.60:40073
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.171.62:37629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.171.62
--webserver_port=37637
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:55:20 UTC on ccycloud.quasar-qwhbmo.root.comops.site
build id 4673737
TSAN enabled
I20260926 07:35:21.620319 11716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:35:21.622432 11716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 07:35:21.635226 11446 tablet_service.cc:1467] Processing CreateTablet for tablet d3f4dd5959ff494882341a2cadbf4def (DEFAULT_TABLE table=default.Table_02fe86b869794353adca0276c7cd9410 [id=12a935394c1f49158f105705ce038532]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 07:35:21.638433 11118 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d3f4dd5959ff494882341a2cadbf4def consensus state has no leader
I20260926 07:35:21.640602 11445 tablet_service.cc:1467] Processing CreateTablet for tablet 4ffd8bef90f14047b367a05295e976dd (DEFAULT_TABLE table=default.Table_dd596536aeae4424879b483396f10f5a [id=6a3b8de693cc4463835fb6e6aba8cad3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:21.641805 11445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ffd8bef90f14047b367a05295e976dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:21.647760 11120 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet f7d22125c5e44b2da8f6f5056565b3f1
W20260926 07:35:21.648355 11117 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 82731550769a416496a451712057bef3 has no consensus state
I20260926 07:35:21.649677 11446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3f4dd5959ff494882341a2cadbf4def. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:21.666404 11119 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet f51c9c04105b4913b890c9929f4b94e7
I20260926 07:35:21.703765 11444 tablet_service.cc:1467] Processing CreateTablet for tablet 82731550769a416496a451712057bef3 (DEFAULT_TABLE table=default.Table_96ce23d5255e41f8a900ac36b2ebfa54 [id=7abe3c8b8e86435d8024abe139d80738]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:21.704977 11444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82731550769a416496a451712057bef3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:21.724802 11744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:35:21.728240 11107 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 82731550769a416496a451712057bef3 (table default.Table_96ce23d5255e41f8a900ac36b2ebfa54 [id=7abe3c8b8e86435d8024abe139d80738], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 82731550769a416496a451712057bef3 consensus state has no leader
I20260926 07:35:21.728929 11591 tablet_service.cc:1467] Processing CreateTablet for tablet d3f4dd5959ff494882341a2cadbf4def (DEFAULT_TABLE table=default.Table_02fe86b869794353adca0276c7cd9410 [id=12a935394c1f49158f105705ce038532]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:21.729558 11590 tablet_service.cc:1467] Processing CreateTablet for tablet 4ffd8bef90f14047b367a05295e976dd (DEFAULT_TABLE table=default.Table_dd596536aeae4424879b483396f10f5a [id=6a3b8de693cc4463835fb6e6aba8cad3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:21.729000 11589 tablet_service.cc:1467] Processing CreateTablet for tablet 82731550769a416496a451712057bef3 (DEFAULT_TABLE table=default.Table_96ce23d5255e41f8a900ac36b2ebfa54 [id=7abe3c8b8e86435d8024abe139d80738]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:21.730832 11591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3f4dd5959ff494882341a2cadbf4def. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:21.731462 11589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82731550769a416496a451712057bef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:21.734386 11590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ffd8bef90f14047b367a05295e976dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:21.738782 11746 tablet_bootstrap.cc:492] T d3f4dd5959ff494882341a2cadbf4def P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:21.746825 11746 tablet_bootstrap.cc:654] T d3f4dd5959ff494882341a2cadbf4def P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:21.749405 11746 log.cc:824] T d3f4dd5959ff494882341a2cadbf4def P 22f3902d01584936b20437606cb86dd1: Log is configured to *not* fsync() on all Append() calls
I20260926 07:35:21.766595 11296 tablet_service.cc:1467] Processing CreateTablet for tablet d3f4dd5959ff494882341a2cadbf4def (DEFAULT_TABLE table=default.Table_02fe86b869794353adca0276c7cd9410 [id=12a935394c1f49158f105705ce038532]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:21.771303 11296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3f4dd5959ff494882341a2cadbf4def. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:21.784981 11119 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_4045e5f8afc744059e82d28c69089dd0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:21.787093 11119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4045e5f8afc744059e82d28c69089dd0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:21.790126 11117 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_6ada159fbff8487c85708620316caa04"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:21.792371 11117 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6ada159fbff8487c85708620316caa04 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:21.796937 11120 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_ccf63d4f690a44f48dfb9bdd72cb8e20"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:21.798987 11120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ccf63d4f690a44f48dfb9bdd72cb8e20 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:21.805084 11118 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_a9688931381e46e3953f583a778c8793"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:21.807039 11118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a9688931381e46e3953f583a778c8793 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:21.820382 11295 tablet_service.cc:1467] Processing CreateTablet for tablet 4ffd8bef90f14047b367a05295e976dd (DEFAULT_TABLE table=default.Table_dd596536aeae4424879b483396f10f5a [id=6a3b8de693cc4463835fb6e6aba8cad3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:21.822067 11295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ffd8bef90f14047b367a05295e976dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:21.825124 11294 tablet_service.cc:1467] Processing CreateTablet for tablet 82731550769a416496a451712057bef3 (DEFAULT_TABLE table=default.Table_96ce23d5255e41f8a900ac36b2ebfa54 [id=7abe3c8b8e86435d8024abe139d80738]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:21.826696 11294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82731550769a416496a451712057bef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:21.910902 11588 tablet_service.cc:1467] Processing CreateTablet for tablet d68ea90e76164b72b8ccf700cacf761f (DEFAULT_TABLE table=default.Table_a9688931381e46e3953f583a778c8793 [id=6cff7c1835b8421f9f220a8ffddfbcfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:21.913009 11444 tablet_service.cc:1467] Processing CreateTablet for tablet d68ea90e76164b72b8ccf700cacf761f (DEFAULT_TABLE table=default.Table_a9688931381e46e3953f583a778c8793 [id=6cff7c1835b8421f9f220a8ffddfbcfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:21.914656 11588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d68ea90e76164b72b8ccf700cacf761f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:21.914649 11444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d68ea90e76164b72b8ccf700cacf761f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:21.923410 11293 tablet_service.cc:1467] Processing CreateTablet for tablet d68ea90e76164b72b8ccf700cacf761f (DEFAULT_TABLE table=default.Table_a9688931381e46e3953f583a778c8793 [id=6cff7c1835b8421f9f220a8ffddfbcfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:21.925057 11293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d68ea90e76164b72b8ccf700cacf761f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:21.935093 11119 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_4045e5f8afc744059e82d28c69089dd0" } modify_external_catalogs: true
I20260926 07:35:21.935635 11119 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_4045e5f8afc744059e82d28c69089dd0" } modify_external_catalogs: true
I20260926 07:35:21.944020 11118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_a9688931381e46e3953f583a778c8793" } modify_external_catalogs: true
I20260926 07:35:21.944693 11118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_a9688931381e46e3953f583a778c8793" } modify_external_catalogs: true
I20260926 07:35:21.955073 11120 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_9d17125d78a24e0e856f15ae94bf3efa" } new_table_name: "default.Table_4bc696441dce4bc687a0344b93eccb9e" modify_external_catalogs: true
I20260926 07:35:21.960996 11117 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_96ce23d5255e41f8a900ac36b2ebfa54" } new_table_name: "default.Table_08513db51d2946ef9d50ba6c7dfa7c2c" modify_external_catalogs: true
I20260926 07:35:22.039206 11118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet d68ea90e76164b72b8ccf700cacf761f
I20260926 07:35:22.043450 11587 tablet_service.cc:1514] Processing DeleteTablet for tablet d68ea90e76164b72b8ccf700cacf761f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:21 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:22.043546 11444 tablet_service.cc:1514] Processing DeleteTablet for tablet d68ea90e76164b72b8ccf700cacf761f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:21 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:22.046433 11292 tablet_service.cc:1514] Processing DeleteTablet for tablet d68ea90e76164b72b8ccf700cacf761f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:21 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:22.051594 11118 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_8f8dd9db65b2430b9ea2a5568780ed1f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:22.055415 11118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f8dd9db65b2430b9ea2a5568780ed1f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:22.058950 11116 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_582802febc8147cba686fb42d81b56df"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:22.061165 11116 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_582802febc8147cba686fb42d81b56df in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:35:22.064337 11107 catalog_manager.cc:4808] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet d68ea90e76164b72b8ccf700cacf761f because the tablet was not found. No further retry: Not found: Tablet not found: d68ea90e76164b72b8ccf700cacf761f
I20260926 07:35:22.065124 11115 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_ac5bc74529664244afa3312bf1bc8cde"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:22.068312 11114 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_2195a1608fc743119e6d69ed4a7564cd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:22.070608 11114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2195a1608fc743119e6d69ed4a7564cd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:35:22.077123 11115 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ac5bc74529664244afa3312bf1bc8cde in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:22.101557 11746 tablet_bootstrap.cc:492] T d3f4dd5959ff494882341a2cadbf4def P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:22.102084 11746 ts_tablet_manager.cc:1397] T d3f4dd5959ff494882341a2cadbf4def P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.364s user 0.117s sys 0.201s
W20260926 07:35:22.134346 11106 catalog_manager.cc:4808] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet d68ea90e76164b72b8ccf700cacf761f because the tablet was not found. No further retry: Not found: Tablet not found: d68ea90e76164b72b8ccf700cacf761f
I20260926 07:35:22.135318 11119 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet e58a6eb33af34202ac9f501d69d769ec
W20260926 07:35:22.150571 11120 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d3f4dd5959ff494882341a2cadbf4def consensus state has no leader
I20260926 07:35:22.151579 11746 raft_consensus.cc:348] T d3f4dd5959ff494882341a2cadbf4def P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:22.152264 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet d68ea90e76164b72b8ccf700cacf761f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d68ea90e76164b72b8ccf700cacf761f already in progress: creating tablet
I20260926 07:35:22.152549 11746 raft_consensus.cc:374] T d3f4dd5959ff494882341a2cadbf4def P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:22.152819 11746 raft_consensus.cc:729] T d3f4dd5959ff494882341a2cadbf4def P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:22.153716 11746 consensus_queue.cc:260] T d3f4dd5959ff494882341a2cadbf4def P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:22.158463 11117 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 82731550769a416496a451712057bef3 consensus state has no leader
I20260926 07:35:22.162825 11511 heartbeater.cc:503] Master 127.10.171.60:40073 was elected leader, sending a full tablet report...
I20260926 07:35:22.168809 11746 ts_tablet_manager.cc:1428] T d3f4dd5959ff494882341a2cadbf4def P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.066s user 0.038s sys 0.000s
I20260926 07:35:22.169057 11363 heartbeater.cc:503] Master 127.10.171.60:40073 was elected leader, sending a full tablet report...
I20260926 07:35:22.177433 11746 tablet_bootstrap.cc:492] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:22.179540 11756 tablet_bootstrap.cc:492] T 4ffd8bef90f14047b367a05295e976dd P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:22.190169 11746 tablet_bootstrap.cc:654] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:22.191694 11119 master_service.cc:946] ReplaceTablet: received request to replace tablet d3f4dd5959ff494882341a2cadbf4def from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:22.195988 11656 heartbeater.cc:503] Master 127.10.171.60:40073 was elected leader, sending a full tablet report...
I20260926 07:35:22.216219 11116 master_service.cc:946] ReplaceTablet: received request to replace tablet 82731550769a416496a451712057bef3 from {username='slave'} at 127.0.0.1:44836
W20260926 07:35:22.219385 11518 tablet.cc:2369] T d3f4dd5959ff494882341a2cadbf4def P 22f3902d01584936b20437606cb86dd1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:35:22.220199 11756 tablet_bootstrap.cc:654] T 4ffd8bef90f14047b367a05295e976dd P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:22.237246 11754 tablet_bootstrap.cc:492] T 82731550769a416496a451712057bef3 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:22.242511 11756 log.cc:824] T 4ffd8bef90f14047b367a05295e976dd P e9bfebb34fa14cd2ad4843fffbe5c54e: Log is configured to *not* fsync() on all Append() calls
I20260926 07:35:22.251978 11746 tablet_bootstrap.cc:492] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:22.252488 11746 ts_tablet_manager.cc:1397] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.075s user 0.015s sys 0.005s
I20260926 07:35:22.255092 11117 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet d3f4dd5959ff494882341a2cadbf4def (table default.Table_4bc696441dce4bc687a0344b93eccb9e [id=12a935394c1f49158f105705ce038532]). Expected version 2 got 0
I20260926 07:35:22.255749 11754 tablet_bootstrap.cc:654] T 82731550769a416496a451712057bef3 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:22.257675 11293 tablet_service.cc:1467] Processing CreateTablet for tablet bd27f6c6ed744d7bbd0c7ab217f0b37e (DEFAULT_TABLE table=default.Table_6ada159fbff8487c85708620316caa04 [id=f232e32225aa4d1a82f4f061434e9f89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.257566 11114 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_6ada159fbff8487c85708620316caa04" } modify_external_catalogs: true
I20260926 07:35:22.258006 11589 tablet_service.cc:1467] Processing CreateTablet for tablet bd27f6c6ed744d7bbd0c7ab217f0b37e (DEFAULT_TABLE table=default.Table_6ada159fbff8487c85708620316caa04 [id=f232e32225aa4d1a82f4f061434e9f89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.258523 11114 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_6ada159fbff8487c85708620316caa04" } modify_external_catalogs: true
I20260926 07:35:22.258893 11293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd27f6c6ed744d7bbd0c7ab217f0b37e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.258998 11589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd27f6c6ed744d7bbd0c7ab217f0b37e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.260483 11588 tablet_service.cc:1467] Processing CreateTablet for tablet 05f071b4db7847a5adbd25a71cac3ea7 (DEFAULT_TABLE table=default.Table_ccf63d4f690a44f48dfb9bdd72cb8e20 [id=9e879069d99f41cdb271157af1a3fd1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.255918 11117 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 82731550769a416496a451712057bef3 (table default.Table_08513db51d2946ef9d50ba6c7dfa7c2c [id=7abe3c8b8e86435d8024abe139d80738]). Expected version 2 got 0
I20260926 07:35:22.261475 11588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05f071b4db7847a5adbd25a71cac3ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.261281 11296 tablet_service.cc:1467] Processing CreateTablet for tablet 05f071b4db7847a5adbd25a71cac3ea7 (DEFAULT_TABLE table=default.Table_ccf63d4f690a44f48dfb9bdd72cb8e20 [id=9e879069d99f41cdb271157af1a3fd1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.262231 11296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05f071b4db7847a5adbd25a71cac3ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.274909 11756 tablet_bootstrap.cc:492] T 4ffd8bef90f14047b367a05295e976dd P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:22.275449 11756 ts_tablet_manager.cc:1397] T 4ffd8bef90f14047b367a05295e976dd P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.096s user 0.007s sys 0.008s
I20260926 07:35:22.277392 11444 tablet_service.cc:1467] Processing CreateTablet for tablet bd27f6c6ed744d7bbd0c7ab217f0b37e (DEFAULT_TABLE table=default.Table_6ada159fbff8487c85708620316caa04 [id=f232e32225aa4d1a82f4f061434e9f89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.278003 11746 raft_consensus.cc:348] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:22.278749 11746 raft_consensus.cc:374] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:22.279088 11746 raft_consensus.cc:729] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:22.279675 11445 tablet_service.cc:1467] Processing CreateTablet for tablet 05f071b4db7847a5adbd25a71cac3ea7 (DEFAULT_TABLE table=default.Table_ccf63d4f690a44f48dfb9bdd72cb8e20 [id=9e879069d99f41cdb271157af1a3fd1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.279765 11746 consensus_queue.cc:260] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:22.281021 11446 tablet_service.cc:1514] Processing DeleteTablet for tablet d68ea90e76164b72b8ccf700cacf761f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:21 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:22.278425 11444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd27f6c6ed744d7bbd0c7ab217f0b37e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.282344 11746 ts_tablet_manager.cc:1428] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
W20260926 07:35:22.287706 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet d68ea90e76164b72b8ccf700cacf761f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d68ea90e76164b72b8ccf700cacf761f already in progress: creating tablet
I20260926 07:35:22.292373 11445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05f071b4db7847a5adbd25a71cac3ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.294068 11746 tablet_bootstrap.cc:492] T 82731550769a416496a451712057bef3 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:22.307539 11746 tablet_bootstrap.cc:654] T 82731550769a416496a451712057bef3 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:22.312268 11116 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 82731550769a416496a451712057bef3
I20260926 07:35:22.314033 11295 tablet_service.cc:1514] Processing DeleteTablet for tablet 82731550769a416496a451712057bef3 with delete_type TABLET_DATA_DELETED (replaced by tablet 1565a4e25ed54bfda4e27f808b67abe0) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:22.314848 11446 tablet_service.cc:1514] Processing DeleteTablet for tablet 82731550769a416496a451712057bef3 with delete_type TABLET_DATA_DELETED (replaced by tablet 1565a4e25ed54bfda4e27f808b67abe0) from {username='slave'} at 127.0.0.1:40244
W20260926 07:35:22.316008 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 82731550769a416496a451712057bef3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 82731550769a416496a451712057bef3 already in progress: creating tablet
W20260926 07:35:22.316661 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 82731550769a416496a451712057bef3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 82731550769a416496a451712057bef3 already in progress: creating tablet
I20260926 07:35:22.319103 11116 catalog_manager.cc:6565] ReplaceTablet: tablet 82731550769a416496a451712057bef3 deleted and replaced by tablet 1565a4e25ed54bfda4e27f808b67abe0
I20260926 07:35:22.319185 11588 tablet_service.cc:1514] Processing DeleteTablet for tablet 82731550769a416496a451712057bef3 with delete_type TABLET_DATA_DELETED (replaced by tablet 1565a4e25ed54bfda4e27f808b67abe0) from {username='slave'} at 127.0.0.1:35244
W20260926 07:35:22.321267 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 82731550769a416496a451712057bef3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 82731550769a416496a451712057bef3 already in progress: creating tablet
I20260926 07:35:22.322029 11746 tablet_bootstrap.cc:492] T 82731550769a416496a451712057bef3 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:22.322419 11746 ts_tablet_manager.cc:1397] T 82731550769a416496a451712057bef3 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.029s user 0.009s sys 0.002s
I20260926 07:35:22.326917 11754 log.cc:824] T 82731550769a416496a451712057bef3 P 6070e9e3d1234a9d83e49431416afade: Log is configured to *not* fsync() on all Append() calls
I20260926 07:35:22.328516 11746 raft_consensus.cc:348] T 82731550769a416496a451712057bef3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:22.329233 11746 raft_consensus.cc:374] T 82731550769a416496a451712057bef3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:22.329505 11746 raft_consensus.cc:729] T 82731550769a416496a451712057bef3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:22.330112 11746 consensus_queue.cc:260] T 82731550769a416496a451712057bef3 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:22.335801 11746 ts_tablet_manager.cc:1428] T 82731550769a416496a451712057bef3 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.013s user 0.001s sys 0.005s
I20260926 07:35:22.339754 11746 tablet_bootstrap.cc:492] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:22.355082 11746 tablet_bootstrap.cc:654] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:22.348308 11756 raft_consensus.cc:348] T 4ffd8bef90f14047b367a05295e976dd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:22.356756 11756 raft_consensus.cc:374] T 4ffd8bef90f14047b367a05295e976dd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:22.357082 11756 raft_consensus.cc:729] T 4ffd8bef90f14047b367a05295e976dd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:22.357326 11115 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_2195a1608fc743119e6d69ed4a7564cd" } new_table_name: "default.Table_6380aa40746f46b49a2e47e8c66e854a" modify_external_catalogs: true
I20260926 07:35:22.357949 11756 consensus_queue.cc:260] T 4ffd8bef90f14047b367a05295e976dd P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:22.361068 11119 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet d3f4dd5959ff494882341a2cadbf4def
I20260926 07:35:22.361730 11118 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_ac5bc74529664244afa3312bf1bc8cde" } new_table_name: "default.Table_cb7f69c004f64db7bdc4412924ca513e" modify_external_catalogs: true
I20260926 07:35:22.363801 11295 tablet_service.cc:1514] Processing DeleteTablet for tablet d3f4dd5959ff494882341a2cadbf4def with delete_type TABLET_DATA_DELETED (replaced by tablet 7000bcd55a984d64ba9d7d50fe63822c) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:22.367700 11295 tablet_service.cc:1514] Processing DeleteTablet for tablet 82731550769a416496a451712057bef3 with delete_type TABLET_DATA_DELETED (replaced by tablet 1565a4e25ed54bfda4e27f808b67abe0) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:22.368151 11756 ts_tablet_manager.cc:1428] T 4ffd8bef90f14047b367a05295e976dd P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.092s user 0.040s sys 0.003s
I20260926 07:35:22.382882 11119 catalog_manager.cc:6565] ReplaceTablet: tablet d3f4dd5959ff494882341a2cadbf4def deleted and replaced by tablet 7000bcd55a984d64ba9d7d50fe63822c
I20260926 07:35:22.383441 11588 tablet_service.cc:1514] Processing DeleteTablet for tablet d3f4dd5959ff494882341a2cadbf4def with delete_type TABLET_DATA_DELETED (replaced by tablet 7000bcd55a984d64ba9d7d50fe63822c) from {username='slave'} at 127.0.0.1:35244
W20260926 07:35:22.385485 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet d3f4dd5959ff494882341a2cadbf4def because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d3f4dd5959ff494882341a2cadbf4def already in progress: creating tablet
I20260926 07:35:22.385926 11446 tablet_service.cc:1514] Processing DeleteTablet for tablet d3f4dd5959ff494882341a2cadbf4def with delete_type TABLET_DATA_DELETED (replaced by tablet 7000bcd55a984d64ba9d7d50fe63822c) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:22.387274 11752 tablet_replica.cc:330] T d3f4dd5959ff494882341a2cadbf4def P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:22.388428 11752 raft_consensus.cc:2229] T d3f4dd5959ff494882341a2cadbf4def P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:22.389087 11752 raft_consensus.cc:2258] T d3f4dd5959ff494882341a2cadbf4def P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:22.394649 11752 ts_tablet_manager.cc:1905] T d3f4dd5959ff494882341a2cadbf4def P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 07:35:22.404498 11118 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cb27378f5ed24e76804b05d5fb1cf441 has no consensus state
I20260926 07:35:22.408591 11118 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_b8d5ad40dd0b48609fb6831f16cd7fb6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:22.409252 11119 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_9499f3a151d84ca5835128c74b0a17c4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:22.410993 11118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b8d5ad40dd0b48609fb6831f16cd7fb6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:22.412727 11746 tablet_bootstrap.cc:492] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:22.413148 11746 ts_tablet_manager.cc:1397] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.074s user 0.005s sys 0.008s
I20260926 07:35:22.413103 11114 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet bd27f6c6ed744d7bbd0c7ab217f0b37e
I20260926 07:35:22.415625 11295 tablet_service.cc:1514] Processing DeleteTablet for tablet bd27f6c6ed744d7bbd0c7ab217f0b37e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:22.416170 11754 tablet_bootstrap.cc:492] T 82731550769a416496a451712057bef3 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:22.416734 11754 ts_tablet_manager.cc:1397] T 82731550769a416496a451712057bef3 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.180s user 0.007s sys 0.012s
I20260926 07:35:22.418977 11588 tablet_service.cc:1514] Processing DeleteTablet for tablet bd27f6c6ed744d7bbd0c7ab217f0b37e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:22.420262 11446 tablet_service.cc:1514] Processing DeleteTablet for tablet bd27f6c6ed744d7bbd0c7ab217f0b37e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:40244
W20260926 07:35:22.420874 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet bd27f6c6ed744d7bbd0c7ab217f0b37e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd27f6c6ed744d7bbd0c7ab217f0b37e already in progress: creating tablet
W20260926 07:35:22.425724 11115 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3c038c45769d4e43b15cb4d1d73a3466 has no consensus state
I20260926 07:35:22.432721 11756 tablet_bootstrap.cc:492] T 82731550769a416496a451712057bef3 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
W20260926 07:35:22.438760 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet d3f4dd5959ff494882341a2cadbf4def because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d3f4dd5959ff494882341a2cadbf4def already in progress: creating tablet
I20260926 07:35:22.441092 11756 tablet_bootstrap.cc:654] T 82731550769a416496a451712057bef3 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:22.442978 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 82731550769a416496a451712057bef3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 82731550769a416496a451712057bef3 already in progress: creating tablet
W20260926 07:35:22.443590 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet bd27f6c6ed744d7bbd0c7ab217f0b37e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd27f6c6ed744d7bbd0c7ab217f0b37e already in progress: creating tablet
I20260926 07:35:22.424383 11746 raft_consensus.cc:348] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:22.445398 11746 raft_consensus.cc:374] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:22.445698 11746 raft_consensus.cc:729] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:22.446362 11746 consensus_queue.cc:260] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:22.447152 11115 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_17e4f8bb2ea7464fbc563d08108d8325"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:22.449435 11115 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_17e4f8bb2ea7464fbc563d08108d8325 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:22.454625 11756 tablet_bootstrap.cc:492] T 82731550769a416496a451712057bef3 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:22.455025 11756 ts_tablet_manager.cc:1397] T 82731550769a416496a451712057bef3 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.003s
I20260926 07:35:22.459127 11589 tablet_service.cc:1514] Processing DeleteTablet for tablet d68ea90e76164b72b8ccf700cacf761f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:21 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:22.459908 11588 tablet_service.cc:1514] Processing DeleteTablet for tablet 82731550769a416496a451712057bef3 with delete_type TABLET_DATA_DELETED (replaced by tablet 1565a4e25ed54bfda4e27f808b67abe0) from {username='slave'} at 127.0.0.1:35244
W20260926 07:35:22.460693 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet d68ea90e76164b72b8ccf700cacf761f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d68ea90e76164b72b8ccf700cacf761f already in progress: creating tablet
I20260926 07:35:22.460757 11116 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_08513db51d2946ef9d50ba6c7dfa7c2c" } modify_external_catalogs: true
I20260926 07:35:22.461314 11116 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_08513db51d2946ef9d50ba6c7dfa7c2c" } modify_external_catalogs: true
I20260926 07:35:22.461457 11589 tablet_service.cc:1514] Processing DeleteTablet for tablet d3f4dd5959ff494882341a2cadbf4def with delete_type TABLET_DATA_DELETED (replaced by tablet 7000bcd55a984d64ba9d7d50fe63822c) from {username='slave'} at 127.0.0.1:35244
W20260926 07:35:22.462708 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 82731550769a416496a451712057bef3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 82731550769a416496a451712057bef3 already in progress: creating tablet
I20260926 07:35:22.462350 11754 raft_consensus.cc:348] T 82731550769a416496a451712057bef3 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:22.463229 11754 raft_consensus.cc:374] T 82731550769a416496a451712057bef3 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260926 07:35:22.463339 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet d3f4dd5959ff494882341a2cadbf4def because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d3f4dd5959ff494882341a2cadbf4def already in progress: creating tablet
I20260926 07:35:22.463503 11754 raft_consensus.cc:729] T 82731550769a416496a451712057bef3 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:22.462317 11746 ts_tablet_manager.cc:1428] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.049s user 0.004s sys 0.005s
I20260926 07:35:22.464581 11754 consensus_queue.cc:260] T 82731550769a416496a451712057bef3 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:22.469982 11756 raft_consensus.cc:348] T 82731550769a416496a451712057bef3 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:22.472981 11756 raft_consensus.cc:374] T 82731550769a416496a451712057bef3 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:22.473299 11756 raft_consensus.cc:729] T 82731550769a416496a451712057bef3 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:22.474080 11756 consensus_queue.cc:260] T 82731550769a416496a451712057bef3 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:22.475104 11754 ts_tablet_manager.cc:1428] T 82731550769a416496a451712057bef3 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.058s user 0.037s sys 0.004s
I20260926 07:35:22.474763 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_77fe481f19c54d4a99623d65693ba7ba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:22.476322 11113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_77fe481f19c54d4a99623d65693ba7ba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:22.478525 11754 tablet_bootstrap.cc:492] T 4ffd8bef90f14047b367a05295e976dd P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:22.487087 11754 tablet_bootstrap.cc:654] T 4ffd8bef90f14047b367a05295e976dd P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:22.488447 11119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9499f3a151d84ca5835128c74b0a17c4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:22.491163 11756 ts_tablet_manager.cc:1428] T 82731550769a416496a451712057bef3 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.036s user 0.008s sys 0.002s
I20260926 07:35:22.499205 11756 tablet_bootstrap.cc:492] T d3f4dd5959ff494882341a2cadbf4def P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:22.502592 11589 tablet_service.cc:1467] Processing CreateTablet for tablet 3c038c45769d4e43b15cb4d1d73a3466 (DEFAULT_TABLE table=default.Table_6380aa40746f46b49a2e47e8c66e854a [id=f23a7ed0439c41149de764ab244ef60b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.502849 11446 tablet_service.cc:1467] Processing CreateTablet for tablet 3c038c45769d4e43b15cb4d1d73a3466 (DEFAULT_TABLE table=default.Table_6380aa40746f46b49a2e47e8c66e854a [id=f23a7ed0439c41149de764ab244ef60b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.503834 11446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c038c45769d4e43b15cb4d1d73a3466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.503998 11589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c038c45769d4e43b15cb4d1d73a3466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.518424 11588 tablet_service.cc:1467] Processing CreateTablet for tablet a000cb9928de4459be1e6b1b7a81848a (DEFAULT_TABLE table=default.Table_8f8dd9db65b2430b9ea2a5568780ed1f [id=19875953f2b6484f934503404af57593]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.519807 11588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a000cb9928de4459be1e6b1b7a81848a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.521037 11443 tablet_service.cc:1467] Processing CreateTablet for tablet a000cb9928de4459be1e6b1b7a81848a (DEFAULT_TABLE table=default.Table_8f8dd9db65b2430b9ea2a5568780ed1f [id=19875953f2b6484f934503404af57593]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.522076 11443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a000cb9928de4459be1e6b1b7a81848a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.554495 11590 tablet_service.cc:1467] Processing CreateTablet for tablet b7b89fcab5954a3aa1250be4a14a7bda (DEFAULT_TABLE table=default.Table_582802febc8147cba686fb42d81b56df [id=10eb71ca30de4a7da89de44126029c68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.555542 11590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7b89fcab5954a3aa1250be4a14a7bda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.557906 11746 tablet_bootstrap.cc:492] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:22.558807 11591 tablet_service.cc:1467] Processing CreateTablet for tablet cb27378f5ed24e76804b05d5fb1cf441 (DEFAULT_TABLE table=default.Table_cb7f69c004f64db7bdc4412924ca513e [id=4c1477cba48a45558d6e0dfe08cd8afa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.559861 11591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb27378f5ed24e76804b05d5fb1cf441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.557919 11442 tablet_service.cc:1467] Processing CreateTablet for tablet b7b89fcab5954a3aa1250be4a14a7bda (DEFAULT_TABLE table=default.Table_582802febc8147cba686fb42d81b56df [id=10eb71ca30de4a7da89de44126029c68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.561358 11442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7b89fcab5954a3aa1250be4a14a7bda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.563365 11755 raft_consensus.cc:482] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:22.564002 11755 raft_consensus.cc:504] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:22.573993 11116 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 1565a4e25ed54bfda4e27f808b67abe0
I20260926 07:35:22.581568 11587 tablet_service.cc:1514] Processing DeleteTablet for tablet 82731550769a416496a451712057bef3 with delete_type TABLET_DATA_DELETED (replaced by tablet 1565a4e25ed54bfda4e27f808b67abe0) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:22.582376 11751 tablet_replica.cc:330] T 82731550769a416496a451712057bef3 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:22.583158 11751 raft_consensus.cc:2229] T 82731550769a416496a451712057bef3 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:22.583694 11751 raft_consensus.cc:2258] T 82731550769a416496a451712057bef3 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:22.590302 11441 tablet_service.cc:1467] Processing CreateTablet for tablet cb27378f5ed24e76804b05d5fb1cf441 (DEFAULT_TABLE table=default.Table_cb7f69c004f64db7bdc4412924ca513e [id=4c1477cba48a45558d6e0dfe08cd8afa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.591931 11441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb27378f5ed24e76804b05d5fb1cf441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.595500 11440 tablet_service.cc:1514] Processing DeleteTablet for tablet 82731550769a416496a451712057bef3 with delete_type TABLET_DATA_DELETED (replaced by tablet 1565a4e25ed54bfda4e27f808b67abe0) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:22.604274 11751 ts_tablet_manager.cc:1905] T 82731550769a416496a451712057bef3 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:22.603999 11755 leader_election.cc:288] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:22.608403 11294 tablet_service.cc:1467] Processing CreateTablet for tablet cb27378f5ed24e76804b05d5fb1cf441 (DEFAULT_TABLE table=default.Table_cb7f69c004f64db7bdc4412924ca513e [id=4c1477cba48a45558d6e0dfe08cd8afa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.610222 11294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb27378f5ed24e76804b05d5fb1cf441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.610424 11292 tablet_service.cc:1514] Processing DeleteTablet for tablet 82731550769a416496a451712057bef3 with delete_type TABLET_DATA_DELETED (replaced by tablet 1565a4e25ed54bfda4e27f808b67abe0) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:22.612078 11293 tablet_service.cc:1467] Processing CreateTablet for tablet 3c038c45769d4e43b15cb4d1d73a3466 (DEFAULT_TABLE table=default.Table_6380aa40746f46b49a2e47e8c66e854a [id=f23a7ed0439c41149de764ab244ef60b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.613143 11293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c038c45769d4e43b15cb4d1d73a3466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.617853 11113 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_9499f3a151d84ca5835128c74b0a17c4" } new_table_name: "default.Table_836a28372ec4439d98b1583a9b50d2c1" modify_external_catalogs: true
I20260926 07:35:22.618412 11295 tablet_service.cc:1467] Processing CreateTablet for tablet b7b89fcab5954a3aa1250be4a14a7bda (DEFAULT_TABLE table=default.Table_582802febc8147cba686fb42d81b56df [id=10eb71ca30de4a7da89de44126029c68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.619516 11295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7b89fcab5954a3aa1250be4a14a7bda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.623836 11114 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_b8d5ad40dd0b48609fb6831f16cd7fb6" } modify_external_catalogs: true
I20260926 07:35:22.624328 11296 tablet_service.cc:1467] Processing CreateTablet for tablet a000cb9928de4459be1e6b1b7a81848a (DEFAULT_TABLE table=default.Table_8f8dd9db65b2430b9ea2a5568780ed1f [id=19875953f2b6484f934503404af57593]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.624512 11114 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_b8d5ad40dd0b48609fb6831f16cd7fb6" } modify_external_catalogs: true
I20260926 07:35:22.625336 11296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a000cb9928de4459be1e6b1b7a81848a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.629197 11120 master_service.cc:946] ReplaceTablet: received request to replace tablet d68ea90e76164b72b8ccf700cacf761f from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:22.637844 11118 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_8f8dd9db65b2430b9ea2a5568780ed1f" } new_table_name: "default.Table_0e49a5d925a54102bcf17aea27ffc902" modify_external_catalogs: true
I20260926 07:35:22.640601 11120 master_service.cc:946] ReplaceTablet: received request to replace tablet 82731550769a416496a451712057bef3 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:22.662149 11754 tablet_bootstrap.cc:492] T 4ffd8bef90f14047b367a05295e976dd P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:22.662707 11754 ts_tablet_manager.cc:1397] T 4ffd8bef90f14047b367a05295e976dd P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.184s user 0.046s sys 0.035s
I20260926 07:35:22.665880 11754 raft_consensus.cc:348] T 4ffd8bef90f14047b367a05295e976dd P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:22.667907 11754 raft_consensus.cc:374] T 4ffd8bef90f14047b367a05295e976dd P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:22.668258 11754 raft_consensus.cc:729] T 4ffd8bef90f14047b367a05295e976dd P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:22.673352 11746 tablet_bootstrap.cc:654] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:22.673689 11446 tablet_service.cc:1467] Processing CreateTablet for tablet 8cc53f05536743b8a24ab2c88937f8f2 (DEFAULT_TABLE table=default.Table_9499f3a151d84ca5835128c74b0a17c4 [id=800a96cbd0b34639bbe7ad317a640053]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.622834 11444 tablet_service.cc:1514] Processing DeleteTablet for tablet d68ea90e76164b72b8ccf700cacf761f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:21 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:22.675879 11440 tablet_service.cc:1467] Processing CreateTablet for tablet 50d0e4cd192c445bb7c82d1a3c4a53d1 (DEFAULT_TABLE table=default.Table_b8d5ad40dd0b48609fb6831f16cd7fb6 [id=31cfe82ee0424be9bb920fa81e2a22f0]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 07:35:22.676533 11113 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8cc53f05536743b8a24ab2c88937f8f2 consensus state has no leader
I20260926 07:35:22.676872 11754 consensus_queue.cc:260] T 4ffd8bef90f14047b367a05295e976dd P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:22.678844 11292 tablet_service.cc:1467] Processing CreateTablet for tablet 8cc53f05536743b8a24ab2c88937f8f2 (DEFAULT_TABLE table=default.Table_9499f3a151d84ca5835128c74b0a17c4 [id=800a96cbd0b34639bbe7ad317a640053]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.680395 11292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cc53f05536743b8a24ab2c88937f8f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.680801 11589 tablet_service.cc:1467] Processing CreateTablet for tablet 50d0e4cd192c445bb7c82d1a3c4a53d1 (DEFAULT_TABLE table=default.Table_b8d5ad40dd0b48609fb6831f16cd7fb6 [id=31cfe82ee0424be9bb920fa81e2a22f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.680821 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_52c3bebb6bed469880b91bd49e33ca23"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:22.681798 11589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50d0e4cd192c445bb7c82d1a3c4a53d1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:22.682390 11113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_52c3bebb6bed469880b91bd49e33ca23 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:22.684329 11446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cc53f05536743b8a24ab2c88937f8f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.684541 11590 tablet_service.cc:1467] Processing CreateTablet for tablet 8cc53f05536743b8a24ab2c88937f8f2 (DEFAULT_TABLE table=default.Table_9499f3a151d84ca5835128c74b0a17c4 [id=800a96cbd0b34639bbe7ad317a640053]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.685245 11291 tablet_service.cc:1467] Processing CreateTablet for tablet 50d0e4cd192c445bb7c82d1a3c4a53d1 (DEFAULT_TABLE table=default.Table_b8d5ad40dd0b48609fb6831f16cd7fb6 [id=31cfe82ee0424be9bb920fa81e2a22f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.686406 11291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50d0e4cd192c445bb7c82d1a3c4a53d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.685483 11590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cc53f05536743b8a24ab2c88937f8f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.697181 11440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50d0e4cd192c445bb7c82d1a3c4a53d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.710911 11754 ts_tablet_manager.cc:1428] T 4ffd8bef90f14047b367a05295e976dd P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.048s user 0.006s sys 0.000s
I20260926 07:35:22.716570 11290 tablet_service.cc:1467] Processing CreateTablet for tablet 7000bcd55a984d64ba9d7d50fe63822c (DEFAULT_TABLE table=default.Table_4bc696441dce4bc687a0344b93eccb9e [id=12a935394c1f49158f105705ce038532]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.717809 11290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7000bcd55a984d64ba9d7d50fe63822c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.720963 11114 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 50d0e4cd192c445bb7c82d1a3c4a53d1
I20260926 07:35:22.721994 11752 ts_tablet_manager.cc:1918] T d3f4dd5959ff494882341a2cadbf4def P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:22.722380 11752 log.cc:1196] T d3f4dd5959ff494882341a2cadbf4def P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/d3f4dd5959ff494882341a2cadbf4def
I20260926 07:35:22.724568 11289 tablet_service.cc:1514] Processing DeleteTablet for tablet 50d0e4cd192c445bb7c82d1a3c4a53d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:22.726121 11591 tablet_service.cc:1514] Processing DeleteTablet for tablet 50d0e4cd192c445bb7c82d1a3c4a53d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:22.731377 11754 tablet_bootstrap.cc:492] T d3f4dd5959ff494882341a2cadbf4def P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:22.740906 11591 tablet_service.cc:1467] Processing CreateTablet for tablet 7000bcd55a984d64ba9d7d50fe63822c (DEFAULT_TABLE table=default.Table_4bc696441dce4bc687a0344b93eccb9e [id=12a935394c1f49158f105705ce038532]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.742089 11591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7000bcd55a984d64ba9d7d50fe63822c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.751564 11765 raft_consensus.cc:482] T 4ffd8bef90f14047b367a05295e976dd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:22.752682 11753 tablet_replica.cc:330] T 82731550769a416496a451712057bef3 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
W20260926 07:35:22.751864 11371 tablet.cc:2369] T 4ffd8bef90f14047b367a05295e976dd P e9bfebb34fa14cd2ad4843fffbe5c54e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:35:22.752303 11765 raft_consensus.cc:504] T 4ffd8bef90f14047b367a05295e976dd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:22.754323 11756 tablet_bootstrap.cc:654] T d3f4dd5959ff494882341a2cadbf4def P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:22.760499 11751 ts_tablet_manager.cc:1918] T 82731550769a416496a451712057bef3 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:22.759730 11120 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_e2ca52cf31c34e1cb2e1cf4bd05510c0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:22.762746 11120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e2ca52cf31c34e1cb2e1cf4bd05510c0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:22.761154 11751 log.cc:1196] T 82731550769a416496a451712057bef3 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/82731550769a416496a451712057bef3
W20260926 07:35:22.763983 11105 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet cb27378f5ed24e76804b05d5fb1cf441 (table default.Table_cb7f69c004f64db7bdc4412924ca513e [id=4c1477cba48a45558d6e0dfe08cd8afa], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cb27378f5ed24e76804b05d5fb1cf441 consensus state has no leader
I20260926 07:35:22.764443 11751 ts_tablet_manager.cc:1939] T 82731550769a416496a451712057bef3 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
W20260926 07:35:22.769924 11118 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a000cb9928de4459be1e6b1b7a81848a consensus state has no leader
I20260926 07:35:22.776768 11118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_cb7f69c004f64db7bdc4412924ca513e" } modify_external_catalogs: true
I20260926 07:35:22.777281 11118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_cb7f69c004f64db7bdc4412924ca513e" } modify_external_catalogs: true
I20260926 07:35:22.780018 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_ff479f29ec0143d6a74b38e25389f1bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:22.781562 11113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff479f29ec0143d6a74b38e25389f1bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:22.783664 11443 tablet_service.cc:1514] Processing DeleteTablet for tablet 50d0e4cd192c445bb7c82d1a3c4a53d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:22.788679 11752 ts_tablet_manager.cc:1939] T d3f4dd5959ff494882341a2cadbf4def P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:22.799345 11754 tablet_bootstrap.cc:654] T d3f4dd5959ff494882341a2cadbf4def P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:22.803076 11120 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_c24531decb2643edb1b85ebbcc0c60ba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:22.805430 11120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c24531decb2643edb1b85ebbcc0c60ba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:22.813730 11118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet cb27378f5ed24e76804b05d5fb1cf441
I20260926 07:35:22.768230 11753 raft_consensus.cc:2229] T 82731550769a416496a451712057bef3 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:22.824877 11753 raft_consensus.cc:2258] T 82731550769a416496a451712057bef3 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:22.826262 11754 tablet_bootstrap.cc:492] T d3f4dd5959ff494882341a2cadbf4def P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:22.826747 11754 ts_tablet_manager.cc:1397] T d3f4dd5959ff494882341a2cadbf4def P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.096s user 0.008s sys 0.004s
I20260926 07:35:22.829432 11754 raft_consensus.cc:348] T d3f4dd5959ff494882341a2cadbf4def P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:22.832783 11753 ts_tablet_manager.cc:1905] T 82731550769a416496a451712057bef3 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:22.833174 11754 raft_consensus.cc:374] T d3f4dd5959ff494882341a2cadbf4def P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:22.833563 11754 raft_consensus.cc:729] T d3f4dd5959ff494882341a2cadbf4def P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:22.834571 11754 consensus_queue.cc:260] T d3f4dd5959ff494882341a2cadbf4def P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:22.770704 11765 leader_election.cc:288] T 4ffd8bef90f14047b367a05295e976dd P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:22.837381 11746 tablet_bootstrap.cc:492] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:22.837765 11746 ts_tablet_manager.cc:1397] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.281s user 0.008s sys 0.011s
I20260926 07:35:22.841488 11752 tablet_replica.cc:330] T 82731550769a416496a451712057bef3 P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:22.842258 11752 raft_consensus.cc:2229] T 82731550769a416496a451712057bef3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:22.842696 11752 raft_consensus.cc:2258] T 82731550769a416496a451712057bef3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:22.849591 11754 ts_tablet_manager.cc:1428] T d3f4dd5959ff494882341a2cadbf4def P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20260926 07:35:22.856097 11291 tablet_service.cc:1514] Processing DeleteTablet for tablet cb27378f5ed24e76804b05d5fb1cf441 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:22.860931 11590 tablet_service.cc:1514] Processing DeleteTablet for tablet cb27378f5ed24e76804b05d5fb1cf441 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:22.862967 11754 tablet_bootstrap.cc:492] T d68ea90e76164b72b8ccf700cacf761f P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:22.865501 11755 raft_consensus.cc:482] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:22.866048 11755 raft_consensus.cc:504] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:22.867681 11755 leader_election.cc:288] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:22.867676 11590 tablet_service.cc:1514] Processing DeleteTablet for tablet d3f4dd5959ff494882341a2cadbf4def with delete_type TABLET_DATA_DELETED (replaced by tablet 7000bcd55a984d64ba9d7d50fe63822c) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:22.871246 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet cb27378f5ed24e76804b05d5fb1cf441 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:22.871896 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet d3f4dd5959ff494882341a2cadbf4def (table default.Table_4bc696441dce4bc687a0344b93eccb9e [id=12a935394c1f49158f105705ce038532]) successfully deleted
W20260926 07:35:22.872666 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet bd27f6c6ed744d7bbd0c7ab217f0b37e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd27f6c6ed744d7bbd0c7ab217f0b37e already in progress: creating tablet
I20260926 07:35:22.876868 11440 tablet_service.cc:1467] Processing CreateTablet for tablet 7000bcd55a984d64ba9d7d50fe63822c (DEFAULT_TABLE table=default.Table_4bc696441dce4bc687a0344b93eccb9e [id=12a935394c1f49158f105705ce038532]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:22.878108 11440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7000bcd55a984d64ba9d7d50fe63822c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:22.881543 11752 ts_tablet_manager.cc:1905] T 82731550769a416496a451712057bef3 P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:22.881743 11754 tablet_bootstrap.cc:654] T d68ea90e76164b72b8ccf700cacf761f P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:22.840068 11746 raft_consensus.cc:348] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:22.884965 11746 raft_consensus.cc:374] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:22.885239 11746 raft_consensus.cc:729] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:22.885928 11611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d68ea90e76164b72b8ccf700cacf761f" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:22.887764 11610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ffd8bef90f14047b367a05295e976dd" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:22.888607 11610 raft_consensus.cc:2454] T 4ffd8bef90f14047b367a05295e976dd P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
W20260926 07:35:22.890712 11663 tablet.cc:2369] T 4ffd8bef90f14047b367a05295e976dd P 6070e9e3d1234a9d83e49431416afade: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:35:22.891271 11118 master_service.cc:946] ReplaceTablet: received request to replace tablet 8cc53f05536743b8a24ab2c88937f8f2 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:22.893306 11113 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_52c3bebb6bed469880b91bd49e33ca23" } new_table_name: "default.Table_385be3835b9347ff9e88b710db59a2f6" modify_external_catalogs: true
I20260926 07:35:22.899820 11751 tablet_replica.cc:330] T d3f4dd5959ff494882341a2cadbf4def P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:22.902364 11751 raft_consensus.cc:2229] T d3f4dd5959ff494882341a2cadbf4def P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:22.903338 11751 raft_consensus.cc:2258] T d3f4dd5959ff494882341a2cadbf4def P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:22.905876 11751 ts_tablet_manager.cc:1905] T d3f4dd5959ff494882341a2cadbf4def P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:22.904567 11746 consensus_queue.cc:260] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:22.913807 11756 tablet_bootstrap.cc:492] T d3f4dd5959ff494882341a2cadbf4def P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:22.921409 11756 ts_tablet_manager.cc:1397] T d3f4dd5959ff494882341a2cadbf4def P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.422s user 0.012s sys 0.010s
I20260926 07:35:22.927613 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet bd27f6c6ed744d7bbd0c7ab217f0b37e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:22.928905 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 82731550769a416496a451712057bef3 (table default.Table_08513db51d2946ef9d50ba6c7dfa7c2c [id=7abe3c8b8e86435d8024abe139d80738]) successfully deleted
I20260926 07:35:22.924088 11756 raft_consensus.cc:348] T d3f4dd5959ff494882341a2cadbf4def P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:22.929319 11402 leader_election.cc:341] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:22.929665 11756 raft_consensus.cc:374] T d3f4dd5959ff494882341a2cadbf4def P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:22.929797 11746 ts_tablet_manager.cc:1428] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.092s user 0.006s sys 0.001s
I20260926 07:35:22.929960 11756 raft_consensus.cc:729] T d3f4dd5959ff494882341a2cadbf4def P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:22.930549 11756 consensus_queue.cc:260] T d3f4dd5959ff494882341a2cadbf4def P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:22.930156 11402 leader_election.cc:302] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
W20260926 07:35:22.935753 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 50d0e4cd192c445bb7c82d1a3c4a53d1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 50d0e4cd192c445bb7c82d1a3c4a53d1 already in progress: creating tablet
I20260926 07:35:22.936185 11756 ts_tablet_manager.cc:1428] T d3f4dd5959ff494882341a2cadbf4def P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.014s user 0.003s sys 0.004s
I20260926 07:35:22.937184 11756 tablet_bootstrap.cc:492] T d68ea90e76164b72b8ccf700cacf761f P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
W20260926 07:35:22.937742 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet cb27378f5ed24e76804b05d5fb1cf441 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cb27378f5ed24e76804b05d5fb1cf441 already in progress: creating tablet
I20260926 07:35:22.938617 11755 raft_consensus.cc:2781] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:22.938928 11755 raft_consensus.cc:482] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:22.939257 11755 raft_consensus.cc:3037] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:22.944182 11753 ts_tablet_manager.cc:1918] T 82731550769a416496a451712057bef3 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:22.944566 11746 tablet_bootstrap.cc:492] T 3c038c45769d4e43b15cb4d1d73a3466 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:22.944618 11753 log.cc:1196] T 82731550769a416496a451712057bef3 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/82731550769a416496a451712057bef3
I20260926 07:35:22.945499 11753 ts_tablet_manager.cc:1939] T 82731550769a416496a451712057bef3 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:22.948235 11755 raft_consensus.cc:504] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:22.950081 11755 leader_election.cc:288] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:22.952811 11610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ffd8bef90f14047b367a05295e976dd" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:22.952086 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ffd8bef90f14047b367a05295e976dd" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:22.953352 11610 raft_consensus.cc:3037] T 4ffd8bef90f14047b367a05295e976dd P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:22.953730 11466 raft_consensus.cc:2379] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9bfebb34fa14cd2ad4843fffbe5c54e in current term 1: Already voted for candidate 22f3902d01584936b20437606cb86dd1 in this term.
I20260926 07:35:22.929486 11114 master_service.cc:946] ReplaceTablet: received request to replace tablet 3c038c45769d4e43b15cb4d1d73a3466 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:22.959478 11754 tablet_bootstrap.cc:492] T d68ea90e76164b72b8ccf700cacf761f P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:22.959853 11754 ts_tablet_manager.cc:1397] T d68ea90e76164b72b8ccf700cacf761f P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.097s user 0.006s sys 0.003s
I20260926 07:35:22.962218 11754 raft_consensus.cc:348] T d68ea90e76164b72b8ccf700cacf761f P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:22.962990 11754 raft_consensus.cc:374] T d68ea90e76164b72b8ccf700cacf761f P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:22.963276 11754 raft_consensus.cc:729] T d68ea90e76164b72b8ccf700cacf761f P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:22.963994 11754 consensus_queue.cc:260] T d68ea90e76164b72b8ccf700cacf761f P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:22.967329 11120 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_4bc696441dce4bc687a0344b93eccb9e" } modify_external_catalogs: true
I20260926 07:35:22.972990 11115 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_77fe481f19c54d4a99623d65693ba7ba" } new_table_name: "default.Table_c1c4fe0ecbe840e59d8ba4f659432e0d" modify_external_catalogs: true
I20260926 07:35:22.974622 11316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d68ea90e76164b72b8ccf700cacf761f" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:22.976174 11119 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_23c2afadc1164dfab1f47d7c0b373e5f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:22.977254 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ffd8bef90f14047b367a05295e976dd" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:22.977839 11315 raft_consensus.cc:3037] T 4ffd8bef90f14047b367a05295e976dd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
W20260926 07:35:22.978155 11119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_23c2afadc1164dfab1f47d7c0b373e5f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:22.981065 11316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ffd8bef90f14047b367a05295e976dd" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:22.987064 11315 raft_consensus.cc:2454] T 4ffd8bef90f14047b367a05295e976dd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:22.987452 11611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ffd8bef90f14047b367a05295e976dd" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:22.989142 11249 leader_election.cc:302] T 4ffd8bef90f14047b367a05295e976dd P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade
I20260926 07:35:22.989362 11120 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_4bc696441dce4bc687a0344b93eccb9e" } modify_external_catalogs: true
I20260926 07:35:22.990388 11765 raft_consensus.cc:2726] T 4ffd8bef90f14047b367a05295e976dd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:22.990588 11296 tablet_service.cc:1514] Processing DeleteTablet for tablet d3f4dd5959ff494882341a2cadbf4def with delete_type TABLET_DATA_DELETED (replaced by tablet 7000bcd55a984d64ba9d7d50fe63822c) from {username='slave'} at 127.0.0.1:52070
W20260926 07:35:22.992537 11401 leader_election.cc:341] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:22.995286 11754 ts_tablet_manager.cc:1428] T d68ea90e76164b72b8ccf700cacf761f P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.035s user 0.005s sys 0.000s
I20260926 07:35:22.998171 11591 tablet_service.cc:1514] Processing DeleteTablet for tablet d68ea90e76164b72b8ccf700cacf761f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:21 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:22.992934 11401 leader_election.cc:302] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 22f3902d01584936b20437606cb86dd1; no voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:23.000591 11117 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_b5e830fcbaa44c209d50a84038d3b68f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:23.002326 11401 leader_election.cc:302] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:23.003237 11755 raft_consensus.cc:2726] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:23.003750 11755 raft_consensus.cc:2781] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:23.004200 11755 raft_consensus.cc:686] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
W20260926 07:35:23.004629 11117 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5e830fcbaa44c209d50a84038d3b68f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:23.006930 11754 tablet_bootstrap.cc:492] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:23.012552 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet 82731550769a416496a451712057bef3 (table default.Table_08513db51d2946ef9d50ba6c7dfa7c2c [id=7abe3c8b8e86435d8024abe139d80738]) successfully deleted
W20260926 07:35:23.014307 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 50d0e4cd192c445bb7c82d1a3c4a53d1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 50d0e4cd192c445bb7c82d1a3c4a53d1 already in progress: creating tablet
I20260926 07:35:23.008824 11610 raft_consensus.cc:2454] T 4ffd8bef90f14047b367a05295e976dd P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:23.019555 11751 ts_tablet_manager.cc:1918] T d3f4dd5959ff494882341a2cadbf4def P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260926 07:35:23.021363 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet cb27378f5ed24e76804b05d5fb1cf441 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cb27378f5ed24e76804b05d5fb1cf441 already in progress: creating tablet
I20260926 07:35:23.005483 11755 consensus_queue.cc:237] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:23.024744 11113 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 067ae4ea43f54e88874d93b4393dae27 has no consensus state
I20260926 07:35:23.028913 11591 tablet_service.cc:1467] Processing CreateTablet for tablet aa02661e7531402687f03e86dbc19418 (DEFAULT_TABLE table=default.Table_17e4f8bb2ea7464fbc563d08108d8325 [id=d91b2ce98c4f439e890cbf47ac6c2d11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.030789 11591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa02661e7531402687f03e86dbc19418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.023492 11746 tablet_bootstrap.cc:654] T 3c038c45769d4e43b15cb4d1d73a3466 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:23.036595 11753 tablet_replica.cc:330] T d3f4dd5959ff494882341a2cadbf4def P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:23.037343 11753 raft_consensus.cc:2229] T d3f4dd5959ff494882341a2cadbf4def P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:23.037746 11753 raft_consensus.cc:2258] T d3f4dd5959ff494882341a2cadbf4def P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:23.038401 11756 tablet_bootstrap.cc:654] T d68ea90e76164b72b8ccf700cacf761f P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:23.039983 11442 tablet_service.cc:1467] Processing CreateTablet for tablet aa02661e7531402687f03e86dbc19418 (DEFAULT_TABLE table=default.Table_17e4f8bb2ea7464fbc563d08108d8325 [id=d91b2ce98c4f439e890cbf47ac6c2d11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.043619 11753 ts_tablet_manager.cc:1905] T d3f4dd5959ff494882341a2cadbf4def P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:23.045521 11590 tablet_service.cc:1467] Processing CreateTablet for tablet 345d14fc9eea46ad9b1155d538118ee5 (DEFAULT_TABLE table=default.Table_77fe481f19c54d4a99623d65693ba7ba [id=4c9f2ba8e0db427da572f73f02d4fa9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.046474 11590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 345d14fc9eea46ad9b1155d538118ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.050289 11296 tablet_service.cc:1467] Processing CreateTablet for tablet aa02661e7531402687f03e86dbc19418 (DEFAULT_TABLE table=default.Table_17e4f8bb2ea7464fbc563d08108d8325 [id=d91b2ce98c4f439e890cbf47ac6c2d11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.051431 11296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa02661e7531402687f03e86dbc19418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.052881 11294 tablet_service.cc:1467] Processing CreateTablet for tablet 345d14fc9eea46ad9b1155d538118ee5 (DEFAULT_TABLE table=default.Table_77fe481f19c54d4a99623d65693ba7ba [id=4c9f2ba8e0db427da572f73f02d4fa9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.053812 11294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 345d14fc9eea46ad9b1155d538118ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.055982 11446 tablet_service.cc:1467] Processing CreateTablet for tablet 345d14fc9eea46ad9b1155d538118ee5 (DEFAULT_TABLE table=default.Table_77fe481f19c54d4a99623d65693ba7ba [id=4c9f2ba8e0db427da572f73f02d4fa9e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 07:35:23.058202 11115 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 345d14fc9eea46ad9b1155d538118ee5 consensus state has no leader
I20260926 07:35:23.058565 11446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 345d14fc9eea46ad9b1155d538118ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.064677 11442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa02661e7531402687f03e86dbc19418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.074324 11117 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_c24531decb2643edb1b85ebbcc0c60ba" } modify_external_catalogs: true
I20260926 07:35:23.074788 11117 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_c24531decb2643edb1b85ebbcc0c60ba" } modify_external_catalogs: true
I20260926 07:35:23.019913 11751 log.cc:1196] T d3f4dd5959ff494882341a2cadbf4def P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/d3f4dd5959ff494882341a2cadbf4def
I20260926 07:35:23.088585 11752 ts_tablet_manager.cc:1918] T 82731550769a416496a451712057bef3 P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:23.088718 11751 ts_tablet_manager.cc:1939] T d3f4dd5959ff494882341a2cadbf4def P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:23.088992 11752 log.cc:1196] T 82731550769a416496a451712057bef3 P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/82731550769a416496a451712057bef3
I20260926 07:35:23.089615 11752 ts_tablet_manager.cc:1939] T 82731550769a416496a451712057bef3 P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:23.091280 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet d3f4dd5959ff494882341a2cadbf4def (table default.Table_4bc696441dce4bc687a0344b93eccb9e [id=12a935394c1f49158f105705ce038532]) successfully deleted
I20260926 07:35:23.091806 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet 82731550769a416496a451712057bef3 (table default.Table_08513db51d2946ef9d50ba6c7dfa7c2c [id=7abe3c8b8e86435d8024abe139d80738]) successfully deleted
I20260926 07:35:23.092089 11751 tablet_replica.cc:330] T d68ea90e76164b72b8ccf700cacf761f P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:23.092697 11751 raft_consensus.cc:2229] T d68ea90e76164b72b8ccf700cacf761f P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:23.093145 11751 raft_consensus.cc:2258] T d68ea90e76164b72b8ccf700cacf761f P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:23.095157 11119 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_4cd3f9363e0440dd81c1783ecd6e1214"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:23.096390 11119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4cd3f9363e0440dd81c1783ecd6e1214 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:23.105021 11120 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 7000bcd55a984d64ba9d7d50fe63822c
I20260926 07:35:23.105581 11118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 8cc53f05536743b8a24ab2c88937f8f2
I20260926 07:35:23.107190 11118 catalog_manager.cc:6565] ReplaceTablet: tablet 8cc53f05536743b8a24ab2c88937f8f2 deleted and replaced by tablet 9a255ef54f7e410ca8a779e5d7a118ca
I20260926 07:35:23.109002 11291 tablet_service.cc:1514] Processing DeleteTablet for tablet 7000bcd55a984d64ba9d7d50fe63822c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:23.110718 11290 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cc53f05536743b8a24ab2c88937f8f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 9a255ef54f7e410ca8a779e5d7a118ca) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:23.114794 11114 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 3c038c45769d4e43b15cb4d1d73a3466
I20260926 07:35:23.116232 11114 catalog_manager.cc:6565] ReplaceTablet: tablet 3c038c45769d4e43b15cb4d1d73a3466 deleted and replaced by tablet 1620b507f07c446c8c1811895a097409
I20260926 07:35:23.117789 11441 tablet_service.cc:1514] Processing DeleteTablet for tablet 7000bcd55a984d64ba9d7d50fe63822c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:23.118350 11445 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cc53f05536743b8a24ab2c88937f8f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 9a255ef54f7e410ca8a779e5d7a118ca) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:23.093699 11754 tablet_bootstrap.cc:654] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:23.110219 11589 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cc53f05536743b8a24ab2c88937f8f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 9a255ef54f7e410ca8a779e5d7a118ca) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:23.118654 11588 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c038c45769d4e43b15cb4d1d73a3466 with delete_type TABLET_DATA_DELETED (replaced by tablet 1620b507f07c446c8c1811895a097409) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:23.120455 11440 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c038c45769d4e43b15cb4d1d73a3466 with delete_type TABLET_DATA_DELETED (replaced by tablet 1620b507f07c446c8c1811895a097409) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:23.121119 11290 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c038c45769d4e43b15cb4d1d73a3466 with delete_type TABLET_DATA_DELETED (replaced by tablet 1620b507f07c446c8c1811895a097409) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:23.126263 11117 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 79fe2bf17e4842349bba2cb7a3392ebb
I20260926 07:35:23.126827 11756 tablet_bootstrap.cc:492] T d68ea90e76164b72b8ccf700cacf761f P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:23.127223 11756 ts_tablet_manager.cc:1397] T d68ea90e76164b72b8ccf700cacf761f P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.190s user 0.006s sys 0.005s
I20260926 07:35:23.130110 11751 ts_tablet_manager.cc:1905] T d68ea90e76164b72b8ccf700cacf761f P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:23.131994 11119 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_5975ffa7405849b0858063988326912b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:23.138520 11591 tablet_service.cc:1514] Processing DeleteTablet for tablet 7000bcd55a984d64ba9d7d50fe63822c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:35244
W20260926 07:35:23.139074 11119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5975ffa7405849b0858063988326912b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:23.142361 11756 raft_consensus.cc:348] T d68ea90e76164b72b8ccf700cacf761f P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.163601 11588 tablet_service.cc:1467] Processing CreateTablet for tablet f31a069dc888487ebbbb1cb91e886ad0 (DEFAULT_TABLE table=default.Table_e2ca52cf31c34e1cb2e1cf4bd05510c0 [id=47d4200172794cc78a6f386fee4b15b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.164630 11445 tablet_service.cc:1467] Processing CreateTablet for tablet 14e5da3711fe46c4bb46a7a60f41a581 (DEFAULT_TABLE table=default.Table_ff479f29ec0143d6a74b38e25389f1bc [id=2238dbbde77c45cd8689ba10e6dbf472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.164984 11588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f31a069dc888487ebbbb1cb91e886ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.165614 11445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14e5da3711fe46c4bb46a7a60f41a581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.172698 11441 tablet_service.cc:1467] Processing CreateTablet for tablet f31a069dc888487ebbbb1cb91e886ad0 (DEFAULT_TABLE table=default.Table_e2ca52cf31c34e1cb2e1cf4bd05510c0 [id=47d4200172794cc78a6f386fee4b15b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.172915 11591 tablet_service.cc:1467] Processing CreateTablet for tablet 14e5da3711fe46c4bb46a7a60f41a581 (DEFAULT_TABLE table=default.Table_ff479f29ec0143d6a74b38e25389f1bc [id=2238dbbde77c45cd8689ba10e6dbf472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.172382 11119 catalog_manager.cc:5495] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22f3902d01584936b20437606cb86dd1 (127.10.171.2). New cstate: current_term: 1 leader_uuid: "22f3902d01584936b20437606cb86dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } health_report { overall_health: UNKNOWN } } }
I20260926 07:35:23.174343 11591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14e5da3711fe46c4bb46a7a60f41a581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.176744 11440 tablet_service.cc:1467] Processing CreateTablet for tablet 067ae4ea43f54e88874d93b4393dae27 (DEFAULT_TABLE table=default.Table_385be3835b9347ff9e88b710db59a2f6 [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.177352 11752 tablet_replica.cc:330] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:23.178083 11752 raft_consensus.cc:2229] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:23.178519 11752 raft_consensus.cc:2258] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:23.179422 11754 tablet_bootstrap.cc:492] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:23.179818 11754 ts_tablet_manager.cc:1397] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.173s user 0.003s sys 0.008s
I20260926 07:35:23.182122 11754 raft_consensus.cc:348] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.183238 11754 raft_consensus.cc:374] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:23.183509 11754 raft_consensus.cc:729] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:23.187475 11590 tablet_service.cc:1467] Processing CreateTablet for tablet 067ae4ea43f54e88874d93b4393dae27 (DEFAULT_TABLE table=default.Table_385be3835b9347ff9e88b710db59a2f6 [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.188565 11590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 067ae4ea43f54e88874d93b4393dae27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.189522 11754 consensus_queue.cc:260] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.190124 11441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f31a069dc888487ebbbb1cb91e886ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.199096 11440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 067ae4ea43f54e88874d93b4393dae27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.159811 11290 tablet_service.cc:1467] Processing CreateTablet for tablet 14e5da3711fe46c4bb46a7a60f41a581 (DEFAULT_TABLE table=default.Table_ff479f29ec0143d6a74b38e25389f1bc [id=2238dbbde77c45cd8689ba10e6dbf472]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.161734 11291 tablet_service.cc:1467] Processing CreateTablet for tablet f31a069dc888487ebbbb1cb91e886ad0 (DEFAULT_TABLE table=default.Table_e2ca52cf31c34e1cb2e1cf4bd05510c0 [id=47d4200172794cc78a6f386fee4b15b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.201197 11291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f31a069dc888487ebbbb1cb91e886ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.201538 11756 raft_consensus.cc:374] T d68ea90e76164b72b8ccf700cacf761f P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:23.201826 11756 raft_consensus.cc:729] T d68ea90e76164b72b8ccf700cacf761f P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:23.202443 11756 consensus_queue.cc:260] T d68ea90e76164b72b8ccf700cacf761f P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.205799 11290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14e5da3711fe46c4bb46a7a60f41a581. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.208637 11756 ts_tablet_manager.cc:1428] T d68ea90e76164b72b8ccf700cacf761f P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.081s user 0.007s sys 0.001s
I20260926 07:35:23.209223 11752 ts_tablet_manager.cc:1905] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:23.209486 11756 tablet_bootstrap.cc:492] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:23.214231 11746 tablet_bootstrap.cc:492] T 3c038c45769d4e43b15cb4d1d73a3466 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:23.214627 11746 ts_tablet_manager.cc:1397] T 3c038c45769d4e43b15cb4d1d73a3466 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.270s user 0.006s sys 0.010s
I20260926 07:35:23.229079 11589 tablet_service.cc:1514] Processing DeleteTablet for tablet bd27f6c6ed744d7bbd0c7ab217f0b37e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:23.233896 11589 tablet_service.cc:1467] Processing CreateTablet for tablet a0bdf6a4656e4217b0a7013fddc4cca8 (DEFAULT_TABLE table=default.Table_5975ffa7405849b0858063988326912b [id=3ce63f9fb5ad4b82ba2fbbf2e3678271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.234267 11442 tablet_service.cc:1467] Processing CreateTablet for tablet a0bdf6a4656e4217b0a7013fddc4cca8 (DEFAULT_TABLE table=default.Table_5975ffa7405849b0858063988326912b [id=3ce63f9fb5ad4b82ba2fbbf2e3678271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.234915 11589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0bdf6a4656e4217b0a7013fddc4cca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.236608 11446 tablet_service.cc:1467] Processing CreateTablet for tablet acc5568dee9b473f8f5f1f8bc3b5504e (DEFAULT_TABLE table=default.Table_23c2afadc1164dfab1f47d7c0b373e5f [id=6806510fcf2d4b17960903772e08d49b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.238214 11587 tablet_service.cc:1467] Processing CreateTablet for tablet acc5568dee9b473f8f5f1f8bc3b5504e (DEFAULT_TABLE table=default.Table_23c2afadc1164dfab1f47d7c0b373e5f [id=6806510fcf2d4b17960903772e08d49b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.238430 11443 tablet_service.cc:1467] Processing CreateTablet for tablet 9a255ef54f7e410ca8a779e5d7a118ca (DEFAULT_TABLE table=default.Table_836a28372ec4439d98b1583a9b50d2c1 [id=800a96cbd0b34639bbe7ad317a640053]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.239284 11587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acc5568dee9b473f8f5f1f8bc3b5504e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.239288 11442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0bdf6a4656e4217b0a7013fddc4cca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.241269 11444 tablet_service.cc:1467] Processing CreateTablet for tablet afa901cdc6a84220bf36ee669a955b72 (DEFAULT_TABLE table=default.Table_b5e830fcbaa44c209d50a84038d3b68f [id=be6bd29e04f342cbb7993b18e13d9332]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.242223 11444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afa901cdc6a84220bf36ee669a955b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.242884 11586 tablet_service.cc:1467] Processing CreateTablet for tablet 9a255ef54f7e410ca8a779e5d7a118ca (DEFAULT_TABLE table=default.Table_836a28372ec4439d98b1583a9b50d2c1 [id=800a96cbd0b34639bbe7ad317a640053]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.243925 11586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a255ef54f7e410ca8a779e5d7a118ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.244392 11443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a255ef54f7e410ca8a779e5d7a118ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.246824 11753 ts_tablet_manager.cc:1918] T d3f4dd5959ff494882341a2cadbf4def P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:23.247375 11753 log.cc:1196] T d3f4dd5959ff494882341a2cadbf4def P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/d3f4dd5959ff494882341a2cadbf4def
I20260926 07:35:23.248004 11753 ts_tablet_manager.cc:1939] T d3f4dd5959ff494882341a2cadbf4def P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:23.249924 11439 tablet_service.cc:1467] Processing CreateTablet for tablet 1620b507f07c446c8c1811895a097409 (DEFAULT_TABLE table=default.Table_6380aa40746f46b49a2e47e8c66e854a [id=f23a7ed0439c41149de764ab244ef60b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.251439 11439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1620b507f07c446c8c1811895a097409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.254014 11119 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_ccf63d4f690a44f48dfb9bdd72cb8e20" } new_table_name: "default.Table_8b28cfa8ea704f998eae576de3ee6c92" modify_external_catalogs: true
I20260926 07:35:23.255903 11584 tablet_service.cc:1467] Processing CreateTablet for tablet 1620b507f07c446c8c1811895a097409 (DEFAULT_TABLE table=default.Table_6380aa40746f46b49a2e47e8c66e854a [id=f23a7ed0439c41149de764ab244ef60b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.256991 11584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1620b507f07c446c8c1811895a097409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.260368 11446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acc5568dee9b473f8f5f1f8bc3b5504e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.262046 11585 tablet_service.cc:1467] Processing CreateTablet for tablet afa901cdc6a84220bf36ee669a955b72 (DEFAULT_TABLE table=default.Table_b5e830fcbaa44c209d50a84038d3b68f [id=be6bd29e04f342cbb7993b18e13d9332]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.264566 11585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afa901cdc6a84220bf36ee669a955b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.264453 11117 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_385be3835b9347ff9e88b710db59a2f6" } new_table_name: "default.Table_bc4fc15cb6014a1e86ba2886b2950bae" modify_external_catalogs: true
I20260926 07:35:23.267849 11583 tablet_service.cc:1467] Processing CreateTablet for tablet 4826318c83524f41840e770f33bd3696 (DEFAULT_TABLE table=default.Table_4cd3f9363e0440dd81c1783ecd6e1214 [id=55a6e4b7598043d8bf49512f2542c589]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.269371 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4826318c83524f41840e770f33bd3696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.274127 11114 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_73522bd151c644cd81179073e622d339"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:23.275692 11114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_73522bd151c644cd81179073e622d339 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:23.284266 11746 raft_consensus.cc:348] T 3c038c45769d4e43b15cb4d1d73a3466 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.285027 11746 raft_consensus.cc:374] T 3c038c45769d4e43b15cb4d1d73a3466 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:23.285316 11746 raft_consensus.cc:729] T 3c038c45769d4e43b15cb4d1d73a3466 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:23.290742 11754 ts_tablet_manager.cc:1428] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.111s user 0.004s sys 0.004s
I20260926 07:35:23.291628 11754 tablet_bootstrap.cc:492] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:23.298666 11118 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_f3bd805718ba491ebc23332744c77152"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:23.300312 11118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f3bd805718ba491ebc23332744c77152 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:35:23.305095 11119 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 05f071b4db7847a5adbd25a71cac3ea7 consensus state has no leader
I20260926 07:35:23.317601 11119 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_17e4f8bb2ea7464fbc563d08108d8325" } modify_external_catalogs: true
I20260926 07:35:23.318217 11119 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_17e4f8bb2ea7464fbc563d08108d8325" } modify_external_catalogs: true
I20260926 07:35:23.317831 11746 consensus_queue.cc:260] T 3c038c45769d4e43b15cb4d1d73a3466 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.339082 11746 ts_tablet_manager.cc:1428] T 3c038c45769d4e43b15cb4d1d73a3466 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.124s user 0.007s sys 0.000s
W20260926 07:35:23.324712 11117 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 067ae4ea43f54e88874d93b4393dae27 consensus state has no leader
I20260926 07:35:23.350029 11752 ts_tablet_manager.cc:1918] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:23.350414 11752 log.cc:1196] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/d68ea90e76164b72b8ccf700cacf761f
I20260926 07:35:23.352839 11115 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_836a28372ec4439d98b1583a9b50d2c1" } modify_external_catalogs: true
I20260926 07:35:23.353395 11115 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_836a28372ec4439d98b1583a9b50d2c1" } modify_external_catalogs: true
I20260926 07:35:23.354146 11746 tablet_bootstrap.cc:492] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:23.355825 11751 ts_tablet_manager.cc:1918] T d68ea90e76164b72b8ccf700cacf761f P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:23.249815 11292 tablet_service.cc:1467] Processing CreateTablet for tablet a0bdf6a4656e4217b0a7013fddc4cca8 (DEFAULT_TABLE table=default.Table_5975ffa7405849b0858063988326912b [id=3ce63f9fb5ad4b82ba2fbbf2e3678271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.356449 11111 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_65217d931d594a70b45c7d5c486930ba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:23.358525 11111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65217d931d594a70b45c7d5c486930ba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:23.251184 11293 tablet_service.cc:1467] Processing CreateTablet for tablet acc5568dee9b473f8f5f1f8bc3b5504e (DEFAULT_TABLE table=default.Table_23c2afadc1164dfab1f47d7c0b373e5f [id=6806510fcf2d4b17960903772e08d49b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.372850 11293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acc5568dee9b473f8f5f1f8bc3b5504e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.251333 11289 tablet_service.cc:1467] Processing CreateTablet for tablet afa901cdc6a84220bf36ee669a955b72 (DEFAULT_TABLE table=default.Table_b5e830fcbaa44c209d50a84038d3b68f [id=be6bd29e04f342cbb7993b18e13d9332]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.252270 11295 tablet_service.cc:1467] Processing CreateTablet for tablet 9a255ef54f7e410ca8a779e5d7a118ca (DEFAULT_TABLE table=default.Table_836a28372ec4439d98b1583a9b50d2c1 [id=800a96cbd0b34639bbe7ad317a640053]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.377210 11746 tablet_bootstrap.cc:654] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:23.157243 11294 tablet_service.cc:1467] Processing CreateTablet for tablet 067ae4ea43f54e88874d93b4393dae27 (DEFAULT_TABLE table=default.Table_385be3835b9347ff9e88b710db59a2f6 [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.336407 11296 tablet_service.cc:1514] Processing DeleteTablet for tablet d68ea90e76164b72b8ccf700cacf761f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:21 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:23.378443 11289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afa901cdc6a84220bf36ee669a955b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.341784 11288 tablet_service.cc:1467] Processing CreateTablet for tablet 1620b507f07c446c8c1811895a097409 (DEFAULT_TABLE table=default.Table_6380aa40746f46b49a2e47e8c66e854a [id=f23a7ed0439c41149de764ab244ef60b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.345840 11287 tablet_service.cc:1467] Processing CreateTablet for tablet 4826318c83524f41840e770f33bd3696 (DEFAULT_TABLE table=default.Table_4cd3f9363e0440dd81c1783ecd6e1214 [id=55a6e4b7598043d8bf49512f2542c589]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.369956 11756 tablet_bootstrap.cc:700] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent opening tablet: real 0.119s user 0.001s sys 0.000s
I20260926 07:35:23.384624 11756 tablet_bootstrap.cc:654] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:23.386373 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 7000bcd55a984d64ba9d7d50fe63822c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7000bcd55a984d64ba9d7d50fe63822c already in progress: creating tablet
I20260926 07:35:23.393352 11442 tablet_service.cc:1467] Processing CreateTablet for tablet 4826318c83524f41840e770f33bd3696 (DEFAULT_TABLE table=default.Table_4cd3f9363e0440dd81c1783ecd6e1214 [id=55a6e4b7598043d8bf49512f2542c589]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.394492 11442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4826318c83524f41840e770f33bd3696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.395468 11294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 067ae4ea43f54e88874d93b4393dae27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.396385 11117 master_service.cc:946] ReplaceTablet: received request to replace tablet 345d14fc9eea46ad9b1155d538118ee5 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:23.397128 11752 ts_tablet_manager.cc:1939] T d68ea90e76164b72b8ccf700cacf761f P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:23.399880 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet d68ea90e76164b72b8ccf700cacf761f (table default.Table_a9688931381e46e3953f583a778c8793 [id=6cff7c1835b8421f9f220a8ffddfbcfb]) successfully deleted
W20260926 07:35:23.401589 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 50d0e4cd192c445bb7c82d1a3c4a53d1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 50d0e4cd192c445bb7c82d1a3c4a53d1 already in progress: creating tablet
W20260926 07:35:23.403668 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet cb27378f5ed24e76804b05d5fb1cf441 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cb27378f5ed24e76804b05d5fb1cf441 already in progress: creating tablet
I20260926 07:35:23.404690 11752 tablet_replica.cc:330] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:23.409757 11114 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_7846bbd3f8cd4fbbb54ce45d2e3ce6b9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:23.411309 11114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7846bbd3f8cd4fbbb54ce45d2e3ce6b9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:23.412678 11769 raft_consensus.cc:482] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:23.413511 11113 master_service.cc:946] ReplaceTablet: received request to replace tablet bd27f6c6ed744d7bbd0c7ab217f0b37e from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:23.414141 11292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0bdf6a4656e4217b0a7013fddc4cca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.415587 11753 tablet_replica.cc:330] T d68ea90e76164b72b8ccf700cacf761f P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
W20260926 07:35:23.417483 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 8cc53f05536743b8a24ab2c88937f8f2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8cc53f05536743b8a24ab2c88937f8f2 already in progress: creating tablet
W20260926 07:35:23.418177 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 3c038c45769d4e43b15cb4d1d73a3466 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3c038c45769d4e43b15cb4d1d73a3466 already in progress: creating tablet
I20260926 07:35:23.419863 11753 raft_consensus.cc:2229] T d68ea90e76164b72b8ccf700cacf761f P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:23.420778 11753 raft_consensus.cc:2258] T d68ea90e76164b72b8ccf700cacf761f P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:23.420954 11295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a255ef54f7e410ca8a779e5d7a118ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.423009 11753 ts_tablet_manager.cc:1905] T d68ea90e76164b72b8ccf700cacf761f P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:23.413137 11769 raft_consensus.cc:504] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.379035 11751 log.cc:1196] T d68ea90e76164b72b8ccf700cacf761f P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/d68ea90e76164b72b8ccf700cacf761f
I20260926 07:35:23.428727 11752 raft_consensus.cc:2229] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:23.429193 11752 raft_consensus.cc:2258] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:23.432684 11769 leader_election.cc:288] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:23.444947 11751 ts_tablet_manager.cc:1939] T d68ea90e76164b72b8ccf700cacf761f P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:23.428689 11754 tablet_bootstrap.cc:654] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:23.450389 11746 tablet_bootstrap.cc:492] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:23.450728 11746 ts_tablet_manager.cc:1397] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.104s user 0.005s sys 0.017s
I20260926 07:35:23.455435 11746 raft_consensus.cc:348] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.459404 11288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1620b507f07c446c8c1811895a097409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.456067 11746 raft_consensus.cc:374] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:23.466135 11287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4826318c83524f41840e770f33bd3696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.464974 11746 raft_consensus.cc:729] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:23.467604 11746 consensus_queue.cc:260] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.470935 11119 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet aa02661e7531402687f03e86dbc19418
I20260926 07:35:23.477535 11289 tablet_service.cc:1514] Processing DeleteTablet for tablet aa02661e7531402687f03e86dbc19418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:23 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:23.478214 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet aa02661e7531402687f03e86dbc19418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:23 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:23.479683 11746 ts_tablet_manager.cc:1428] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.029s user 0.007s sys 0.000s
I20260926 07:35:23.491969 11752 ts_tablet_manager.cc:1905] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:23.490846 11746 tablet_bootstrap.cc:492] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:23.502373 11119 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet a000cb9928de4459be1e6b1b7a81848a (table default.Table_0e49a5d925a54102bcf17aea27ffc902 [id=19875953f2b6484f934503404af57593]). Expected version 1 got 0
I20260926 07:35:23.503176 11746 tablet_bootstrap.cc:654] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:23.509091 11118 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_0e49a5d925a54102bcf17aea27ffc902" } new_table_name: "default.Table_d0380e9d3f26439bbc91cf476aeae69f" modify_external_catalogs: true
I20260926 07:35:23.514827 11746 tablet_bootstrap.cc:492] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:23.515205 11746 ts_tablet_manager.cc:1397] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.005s
I20260926 07:35:23.514746 11113 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet a000cb9928de4459be1e6b1b7a81848a (table default.Table_0e49a5d925a54102bcf17aea27ffc902 [id=19875953f2b6484f934503404af57593]). Expected version 1 got 0
I20260926 07:35:23.457051 11751 tablet_replica.cc:330] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:23.520193 11746 raft_consensus.cc:348] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.520869 11746 raft_consensus.cc:374] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:23.521113 11746 raft_consensus.cc:729] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:23.527333 11115 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 9a255ef54f7e410ca8a779e5d7a118ca
I20260926 07:35:23.527407 11746 consensus_queue.cc:260] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.530200 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a255ef54f7e410ca8a779e5d7a118ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:23 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:23.534227 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd27f6c6ed744d7bbd0c7ab217f0b37e" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:23.535511 11294 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a255ef54f7e410ca8a779e5d7a118ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:23 UTC) from {username='slave'} at 127.0.0.1:52070
W20260926 07:35:23.535871 11546 leader_election.cc:341] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:23.538157 11746 ts_tablet_manager.cc:1428] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
I20260926 07:35:23.538183 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet d68ea90e76164b72b8ccf700cacf761f (table default.Table_a9688931381e46e3953f583a778c8793 [id=6cff7c1835b8421f9f220a8ffddfbcfb]) successfully deleted
I20260926 07:35:23.525792 11751 raft_consensus.cc:2229] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus shutting down.
W20260926 07:35:23.540966 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 3c038c45769d4e43b15cb4d1d73a3466 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3c038c45769d4e43b15cb4d1d73a3466 already in progress: creating tablet
W20260926 07:35:23.544673 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 7000bcd55a984d64ba9d7d50fe63822c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7000bcd55a984d64ba9d7d50fe63822c already in progress: creating tablet
I20260926 07:35:23.548712 11585 tablet_service.cc:1514] Processing DeleteTablet for tablet aa02661e7531402687f03e86dbc19418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:23 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:23.546200 11751 raft_consensus.cc:2258] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:23.554081 11111 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_4cd3f9363e0440dd81c1783ecd6e1214" } new_table_name: "default.Table_bbac1fe8bf474590ad44d694788a6c3a" modify_external_catalogs: true
I20260926 07:35:23.556596 11115 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_b413c386b9e346668b23e2ad025e2b7b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:23.558065 11115 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b413c386b9e346668b23e2ad025e2b7b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:23.558928 11585 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a255ef54f7e410ca8a779e5d7a118ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:23 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:23.560567 11751 ts_tablet_manager.cc:1905] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:23.545310 11746 tablet_bootstrap.cc:492] T cb27378f5ed24e76804b05d5fb1cf441 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:23.566268 11756 tablet_bootstrap.cc:492] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:23.566646 11756 ts_tablet_manager.cc:1397] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.357s user 0.008s sys 0.009s
I20260926 07:35:23.567646 11753 ts_tablet_manager.cc:1918] T d68ea90e76164b72b8ccf700cacf761f P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:23.567974 11753 log.cc:1196] T d68ea90e76164b72b8ccf700cacf761f P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/d68ea90e76164b72b8ccf700cacf761f
I20260926 07:35:23.568655 11753 ts_tablet_manager.cc:1939] T d68ea90e76164b72b8ccf700cacf761f P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:23.571244 11754 tablet_bootstrap.cc:492] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:23.571610 11754 ts_tablet_manager.cc:1397] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.280s user 0.003s sys 0.007s
I20260926 07:35:23.574234 11754 raft_consensus.cc:348] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.575205 11754 raft_consensus.cc:374] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:23.575536 11754 raft_consensus.cc:729] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:23.576213 11754 consensus_queue.cc:260] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.576658 11756 raft_consensus.cc:348] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.577320 11756 raft_consensus.cc:374] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:23.577576 11756 raft_consensus.cc:729] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:23.578023 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet d68ea90e76164b72b8ccf700cacf761f (table default.Table_a9688931381e46e3953f583a778c8793 [id=6cff7c1835b8421f9f220a8ffddfbcfb]) successfully deleted
I20260926 07:35:23.578714 11754 ts_tablet_manager.cc:1428] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260926 07:35:23.579551 11754 tablet_bootstrap.cc:492] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
W20260926 07:35:23.579617 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet aa02661e7531402687f03e86dbc19418 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa02661e7531402687f03e86dbc19418 already in progress: creating tablet
W20260926 07:35:23.580744 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 9a255ef54f7e410ca8a779e5d7a118ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9a255ef54f7e410ca8a779e5d7a118ca already in progress: creating tablet
I20260926 07:35:23.588001 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd27f6c6ed744d7bbd0c7ab217f0b37e" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
W20260926 07:35:23.589694 11545 leader_election.cc:341] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: Tablet not RUNNING: STOPPED
I20260926 07:35:23.590059 11545 leader_election.cc:302] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
W20260926 07:35:23.590814 11545 raft_consensus.cc:2670] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260926 07:35:23.592273 11751 ts_tablet_manager.cc:1918] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:23.592659 11751 log.cc:1196] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/bd27f6c6ed744d7bbd0c7ab217f0b37e
I20260926 07:35:23.593259 11751 ts_tablet_manager.cc:1939] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:23.591791 11756 consensus_queue.cc:260] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.597067 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet bd27f6c6ed744d7bbd0c7ab217f0b37e (table default.Table_6ada159fbff8487c85708620316caa04 [id=f232e32225aa4d1a82f4f061434e9f89]) successfully deleted
I20260926 07:35:23.599592 11756 ts_tablet_manager.cc:1428] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.033s user 0.004s sys 0.003s
W20260926 07:35:23.599737 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet aa02661e7531402687f03e86dbc19418 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa02661e7531402687f03e86dbc19418 already in progress: creating tablet
W20260926 07:35:23.605917 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 9a255ef54f7e410ca8a779e5d7a118ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9a255ef54f7e410ca8a779e5d7a118ca already in progress: creating tablet
I20260926 07:35:23.607328 11754 tablet_bootstrap.cc:654] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:23.627274 11754 tablet_bootstrap.cc:492] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:23.627678 11754 ts_tablet_manager.cc:1397] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.048s user 0.005s sys 0.005s
I20260926 07:35:23.630286 11754 raft_consensus.cc:348] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.630909 11754 raft_consensus.cc:374] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:23.631173 11754 raft_consensus.cc:729] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:23.631789 11754 consensus_queue.cc:260] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.631484 11756 tablet_bootstrap.cc:492] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:23.633764 11746 tablet_bootstrap.cc:654] T cb27378f5ed24e76804b05d5fb1cf441 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:23.634248 11754 ts_tablet_manager.cc:1428] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 07:35:23.635098 11754 tablet_bootstrap.cc:492] T cb27378f5ed24e76804b05d5fb1cf441 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:23.639956 11120 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_1e052cd91c654d85a0fa1da50d394b29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:23.641321 11120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e052cd91c654d85a0fa1da50d394b29 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:23.648873 11117 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 345d14fc9eea46ad9b1155d538118ee5
I20260926 07:35:23.650612 11117 catalog_manager.cc:6565] ReplaceTablet: tablet 345d14fc9eea46ad9b1155d538118ee5 deleted and replaced by tablet 97d02d87a4d34974b9f1d5409511f62c
I20260926 07:35:23.658165 11756 tablet_bootstrap.cc:654] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:23.658977 11288 tablet_service.cc:1514] Processing DeleteTablet for tablet 345d14fc9eea46ad9b1155d538118ee5 with delete_type TABLET_DATA_DELETED (replaced by tablet 97d02d87a4d34974b9f1d5409511f62c) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:23.659998 11585 tablet_service.cc:1514] Processing DeleteTablet for tablet 345d14fc9eea46ad9b1155d538118ee5 with delete_type TABLET_DATA_DELETED (replaced by tablet 97d02d87a4d34974b9f1d5409511f62c) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:23.660789 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet 345d14fc9eea46ad9b1155d538118ee5 with delete_type TABLET_DATA_DELETED (replaced by tablet 97d02d87a4d34974b9f1d5409511f62c) from {username='slave'} at 127.0.0.1:40244
W20260926 07:35:23.663166 11111 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4826318c83524f41840e770f33bd3696 consensus state has no leader
W20260926 07:35:23.666329 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 345d14fc9eea46ad9b1155d538118ee5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 345d14fc9eea46ad9b1155d538118ee5 already in progress: creating tablet
W20260926 07:35:23.671854 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 345d14fc9eea46ad9b1155d538118ee5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 345d14fc9eea46ad9b1155d538118ee5 already in progress: creating tablet
I20260926 07:35:23.673024 11112 master_service.cc:946] ReplaceTablet: received request to replace tablet 345d14fc9eea46ad9b1155d538118ee5 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:23.673128 11754 tablet_bootstrap.cc:654] T cb27378f5ed24e76804b05d5fb1cf441 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:23.680794 11118 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a000cb9928de4459be1e6b1b7a81848a consensus state has no leader
I20260926 07:35:23.687463 11111 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_c1c4fe0ecbe840e59d8ba4f659432e0d" } modify_external_catalogs: true
I20260926 07:35:23.688573 11111 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_c1c4fe0ecbe840e59d8ba4f659432e0d" } modify_external_catalogs: true
I20260926 07:35:23.699347 11765 raft_consensus.cc:482] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:23.699832 11765 raft_consensus.cc:504] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.701834 11765 leader_election.cc:288] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:23.702981 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05f071b4db7847a5adbd25a71cac3ea7" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:23.703698 11466 raft_consensus.cc:2454] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 0.
I20260926 07:35:23.706475 11610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05f071b4db7847a5adbd25a71cac3ea7" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:23.707070 11610 raft_consensus.cc:2454] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 0.
I20260926 07:35:23.708293 11249 leader_election.cc:302] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:23.710702 11765 raft_consensus.cc:2781] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:23.712986 11765 raft_consensus.cc:482] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:23.713402 11765 raft_consensus.cc:3037] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:23.713615 11752 ts_tablet_manager.cc:1918] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:23.722334 11752 log.cc:1196] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/bd27f6c6ed744d7bbd0c7ab217f0b37e
I20260926 07:35:23.723042 11752 ts_tablet_manager.cc:1939] T bd27f6c6ed744d7bbd0c7ab217f0b37e P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:23.731108 11119 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 05f071b4db7847a5adbd25a71cac3ea7 (table default.Table_8b28cfa8ea704f998eae576de3ee6c92 [id=9e879069d99f41cdb271157af1a3fd1e]). Expected version 1 got 0
I20260926 07:35:23.730990 11288 tablet_service.cc:1467] Processing CreateTablet for tablet e06ed286144941269b9a0c0c60bff0ce (DEFAULT_TABLE table=default.Table_f3bd805718ba491ebc23332744c77152 [id=bad324bd4f1e469ab9435b7a34c83cc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.732148 11288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e06ed286144941269b9a0c0c60bff0ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.733115 11756 tablet_bootstrap.cc:492] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:23.733529 11756 ts_tablet_manager.cc:1397] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.102s user 0.008s sys 0.005s
I20260926 07:35:23.720323 11746 tablet_bootstrap.cc:492] T cb27378f5ed24e76804b05d5fb1cf441 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:23.735917 11746 ts_tablet_manager.cc:1397] T cb27378f5ed24e76804b05d5fb1cf441 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.191s user 0.009s sys 0.003s
I20260926 07:35:23.735956 11756 raft_consensus.cc:348] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.736649 11111 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 97d02d87a4d34974b9f1d5409511f62c
I20260926 07:35:23.736657 11756 raft_consensus.cc:374] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:23.736927 11756 raft_consensus.cc:729] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:23.738251 11112 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_23c2afadc1164dfab1f47d7c0b373e5f" } modify_external_catalogs: true
I20260926 07:35:23.738552 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet bd27f6c6ed744d7bbd0c7ab217f0b37e (table default.Table_6ada159fbff8487c85708620316caa04 [id=f232e32225aa4d1a82f4f061434e9f89]) successfully deleted
I20260926 07:35:23.738802 11112 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_23c2afadc1164dfab1f47d7c0b373e5f" } modify_external_catalogs: true
I20260926 07:35:23.741307 11111 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_482b512cdaa04fef87180f6c8f787245"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:23.743468 11756 consensus_queue.cc:260] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.746330 11746 raft_consensus.cc:348] T cb27378f5ed24e76804b05d5fb1cf441 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:23.746980 11746 raft_consensus.cc:374] T cb27378f5ed24e76804b05d5fb1cf441 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260926 07:35:23.749138 11111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_482b512cdaa04fef87180f6c8f787245 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:23.747161 11116 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet 05f071b4db7847a5adbd25a71cac3ea7 (table default.Table_8b28cfa8ea704f998eae576de3ee6c92 [id=9e879069d99f41cdb271157af1a3fd1e]). Expected version 1 got 0
I20260926 07:35:23.753410 11113 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 05f071b4db7847a5adbd25a71cac3ea7 (table default.Table_8b28cfa8ea704f998eae576de3ee6c92 [id=9e879069d99f41cdb271157af1a3fd1e]). Expected version 1 got 0
W20260926 07:35:23.756579 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 7000bcd55a984d64ba9d7d50fe63822c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7000bcd55a984d64ba9d7d50fe63822c already in progress: creating tablet
I20260926 07:35:23.760150 11119 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 05f071b4db7847a5adbd25a71cac3ea7 (table default.Table_8b28cfa8ea704f998eae576de3ee6c92 [id=9e879069d99f41cdb271157af1a3fd1e]). Expected version 1 got 0
I20260926 07:35:23.747233 11746 raft_consensus.cc:729] T cb27378f5ed24e76804b05d5fb1cf441 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:23.762956 11752 tablet_replica.cc:330] T 3c038c45769d4e43b15cb4d1d73a3466 P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:23.763688 11752 raft_consensus.cc:2229] T 3c038c45769d4e43b15cb4d1d73a3466 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:23.764159 11752 raft_consensus.cc:2258] T 3c038c45769d4e43b15cb4d1d73a3466 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:23.764592 11756 ts_tablet_manager.cc:1428] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.031s user 0.004s sys 0.008s
I20260926 07:35:23.764794 11585 tablet_service.cc:1467] Processing CreateTablet for tablet e06ed286144941269b9a0c0c60bff0ce (DEFAULT_TABLE table=default.Table_f3bd805718ba491ebc23332744c77152 [id=bad324bd4f1e469ab9435b7a34c83cc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.765884 11585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e06ed286144941269b9a0c0c60bff0ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.765650 11746 consensus_queue.cc:260] T cb27378f5ed24e76804b05d5fb1cf441 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:23.767879 11756 tablet_bootstrap.cc:492] T 8cc53f05536743b8a24ab2c88937f8f2 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:23.772816 11442 tablet_service.cc:1467] Processing CreateTablet for tablet e06ed286144941269b9a0c0c60bff0ce (DEFAULT_TABLE table=default.Table_f3bd805718ba491ebc23332744c77152 [id=bad324bd4f1e469ab9435b7a34c83cc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.776331 11802 raft_consensus.cc:482] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:23.776881 11802 raft_consensus.cc:504] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.777637 11756 tablet_bootstrap.cc:654] T 8cc53f05536743b8a24ab2c88937f8f2 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:23.779661 11442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e06ed286144941269b9a0c0c60bff0ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.781718 11802 leader_election.cc:288] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
W20260926 07:35:23.783097 11249 leader_election.cc:341] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Not found: Tablet not found: bd27f6c6ed744d7bbd0c7ab217f0b37e
W20260926 07:35:23.784829 11247 leader_election.cc:341] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Not found: Tablet not found: bd27f6c6ed744d7bbd0c7ab217f0b37e
I20260926 07:35:23.785189 11247 leader_election.cc:302] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade
I20260926 07:35:23.785514 11746 ts_tablet_manager.cc:1428] T cb27378f5ed24e76804b05d5fb1cf441 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.041s user 0.005s sys 0.000s
I20260926 07:35:23.787774 11752 ts_tablet_manager.cc:1905] T 3c038c45769d4e43b15cb4d1d73a3466 P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:23.790170 11802 raft_consensus.cc:2726] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:23.800658 11746 tablet_bootstrap.cc:492] T b7b89fcab5954a3aa1250be4a14a7bda P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:23.803412 11765 raft_consensus.cc:504] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.805003 11756 tablet_bootstrap.cc:492] T 8cc53f05536743b8a24ab2c88937f8f2 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:23.805774 11765 leader_election.cc:288] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:23.806463 11610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05f071b4db7847a5adbd25a71cac3ea7" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:23.806999 11610 raft_consensus.cc:3037] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:23.814298 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05f071b4db7847a5adbd25a71cac3ea7" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:23.814814 11466 raft_consensus.cc:3037] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:23.814709 11287 tablet_service.cc:1467] Processing CreateTablet for tablet 75d0b86635944407b83f5ddfa29fcc4d (DEFAULT_TABLE table=default.Table_73522bd151c644cd81179073e622d339 [id=be243e65ad314326b6117ee7dbae43b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.815658 11287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75d0b86635944407b83f5ddfa29fcc4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.817428 11756 ts_tablet_manager.cc:1397] T 8cc53f05536743b8a24ab2c88937f8f2 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.050s user 0.009s sys 0.006s
I20260926 07:35:23.819470 11756 raft_consensus.cc:348] T 8cc53f05536743b8a24ab2c88937f8f2 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.820039 11756 raft_consensus.cc:374] T 8cc53f05536743b8a24ab2c88937f8f2 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:23.820297 11756 raft_consensus.cc:729] T 8cc53f05536743b8a24ab2c88937f8f2 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:23.820828 11756 consensus_queue.cc:260] T 8cc53f05536743b8a24ab2c88937f8f2 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.825045 11441 tablet_service.cc:1467] Processing CreateTablet for tablet 75d0b86635944407b83f5ddfa29fcc4d (DEFAULT_TABLE table=default.Table_73522bd151c644cd81179073e622d339 [id=be243e65ad314326b6117ee7dbae43b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.826020 11441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75d0b86635944407b83f5ddfa29fcc4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.828326 11756 ts_tablet_manager.cc:1428] T 8cc53f05536743b8a24ab2c88937f8f2 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.011s user 0.004s sys 0.001s
I20260926 07:35:23.831032 11746 tablet_bootstrap.cc:654] T b7b89fcab5954a3aa1250be4a14a7bda P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:23.834707 11756 tablet_bootstrap.cc:492] T b7b89fcab5954a3aa1250be4a14a7bda P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:23.843256 11443 tablet_service.cc:1514] Processing DeleteTablet for tablet cb27378f5ed24e76804b05d5fb1cf441 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:23.845304 11756 tablet_bootstrap.cc:654] T b7b89fcab5954a3aa1250be4a14a7bda P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:23.788625 11107 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a000cb9928de4459be1e6b1b7a81848a (table default.Table_d0380e9d3f26439bbc91cf476aeae69f [id=19875953f2b6484f934503404af57593], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet a000cb9928de4459be1e6b1b7a81848a consensus state has no leader
I20260926 07:35:23.853969 11586 tablet_service.cc:1467] Processing CreateTablet for tablet 75d0b86635944407b83f5ddfa29fcc4d (DEFAULT_TABLE table=default.Table_73522bd151c644cd81179073e622d339 [id=be243e65ad314326b6117ee7dbae43b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:23.854982 11586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75d0b86635944407b83f5ddfa29fcc4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:23.863528 11756 tablet_bootstrap.cc:492] T b7b89fcab5954a3aa1250be4a14a7bda P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:23.863885 11756 ts_tablet_manager.cc:1397] T b7b89fcab5954a3aa1250be4a14a7bda P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.030s user 0.005s sys 0.005s
I20260926 07:35:23.866039 11756 raft_consensus.cc:348] T b7b89fcab5954a3aa1250be4a14a7bda P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.866688 11756 raft_consensus.cc:374] T b7b89fcab5954a3aa1250be4a14a7bda P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:23.866933 11756 raft_consensus.cc:729] T b7b89fcab5954a3aa1250be4a14a7bda P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:23.851958 11112 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet acc5568dee9b473f8f5f1f8bc3b5504e
I20260926 07:35:23.867995 11756 consensus_queue.cc:260] T b7b89fcab5954a3aa1250be4a14a7bda P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.868990 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet acc5568dee9b473f8f5f1f8bc3b5504e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:23 UTC) from {username='slave'} at 127.0.0.1:35244
W20260926 07:35:23.870513 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet acc5568dee9b473f8f5f1f8bc3b5504e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet acc5568dee9b473f8f5f1f8bc3b5504e already in progress: creating tablet
I20260926 07:35:23.872363 11756 ts_tablet_manager.cc:1428] T b7b89fcab5954a3aa1250be4a14a7bda P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260926 07:35:23.874289 11119 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet a000cb9928de4459be1e6b1b7a81848a (table default.Table_d0380e9d3f26439bbc91cf476aeae69f [id=19875953f2b6484f934503404af57593]). Expected version 2 got 0
I20260926 07:35:23.875754 11116 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet 05f071b4db7847a5adbd25a71cac3ea7 (table default.Table_8b28cfa8ea704f998eae576de3ee6c92 [id=9e879069d99f41cdb271157af1a3fd1e]). Expected version 1 got 0
I20260926 07:35:23.879104 11120 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_f3bd805718ba491ebc23332744c77152" } new_table_name: "default.Table_22c5272f755c4a659ac71d7a5e1c3a22" modify_external_catalogs: true
I20260926 07:35:23.879757 11756 tablet_bootstrap.cc:492] T 3c038c45769d4e43b15cb4d1d73a3466 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:23.886806 11114 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_de0d2fdff98146f9aede70ab6a51dbec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:23.888314 11114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_de0d2fdff98146f9aede70ab6a51dbec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:23.889999 11756 tablet_bootstrap.cc:654] T 3c038c45769d4e43b15cb4d1d73a3466 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:23.890507 11118 master_service.cc:946] ReplaceTablet: received request to replace tablet b7b89fcab5954a3aa1250be4a14a7bda from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:23.895048 11117 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_6380aa40746f46b49a2e47e8c66e854a" } new_table_name: "default.Table_0a660d82b80c40d8a4c41b9abdec566f" modify_external_catalogs: true
I20260926 07:35:23.903643 11610 raft_consensus.cc:2454] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 1.
I20260926 07:35:23.905087 11249 leader_election.cc:302] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:23.905929 11803 raft_consensus.cc:2781] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:23.906044 11115 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_b98681120d654cb4bfd9fb67e73e3b74"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:23.906324 11803 raft_consensus.cc:686] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Becoming Leader. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Running, Role: LEADER
W20260926 07:35:23.907979 11115 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b98681120d654cb4bfd9fb67e73e3b74 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:23.907618 11803 consensus_queue.cc:237] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.921613 11756 tablet_bootstrap.cc:492] T 3c038c45769d4e43b15cb4d1d73a3466 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:23.921964 11756 ts_tablet_manager.cc:1397] T 3c038c45769d4e43b15cb4d1d73a3466 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.042s user 0.009s sys 0.000s
I20260926 07:35:23.924326 11756 raft_consensus.cc:348] T 3c038c45769d4e43b15cb4d1d73a3466 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.924928 11756 raft_consensus.cc:374] T 3c038c45769d4e43b15cb4d1d73a3466 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:23.925154 11756 raft_consensus.cc:729] T 3c038c45769d4e43b15cb4d1d73a3466 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:23.925863 11756 consensus_queue.cc:260] T 3c038c45769d4e43b15cb4d1d73a3466 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.930843 11756 ts_tablet_manager.cc:1428] T 3c038c45769d4e43b15cb4d1d73a3466 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260926 07:35:23.932073 11756 tablet_bootstrap.cc:492] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:23.942119 11756 tablet_bootstrap.cc:654] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:23.953450 11104 consensus_peers.cc:487] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d -> Peer d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629): Couldn't send request to peer d6ef5b900d0f4e73a358f415efa97249. Status: Network error: Client connection negotiation failed: client connection to 127.10.171.62:37629: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260926 07:35:23.958215 11118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet b7b89fcab5954a3aa1250be4a14a7bda
I20260926 07:35:23.963878 11287 tablet_service.cc:1514] Processing DeleteTablet for tablet acc5568dee9b473f8f5f1f8bc3b5504e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:23 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:23.965870 11118 catalog_manager.cc:6565] ReplaceTablet: tablet b7b89fcab5954a3aa1250be4a14a7bda deleted and replaced by tablet b7837bc0821a4b68972b663606364955
I20260926 07:35:23.966874 11756 tablet_bootstrap.cc:492] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:23.966864 11287 tablet_service.cc:1514] Processing DeleteTablet for tablet b7b89fcab5954a3aa1250be4a14a7bda with delete_type TABLET_DATA_DELETED (replaced by tablet b7837bc0821a4b68972b663606364955) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:23.967332 11756 ts_tablet_manager.cc:1397] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.035s user 0.007s sys 0.003s
I20260926 07:35:23.967636 11753 tablet_replica.cc:330] T b7b89fcab5954a3aa1250be4a14a7bda P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:23.968380 11753 raft_consensus.cc:2229] T b7b89fcab5954a3aa1250be4a14a7bda P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:23.968128 11443 tablet_service.cc:1514] Processing DeleteTablet for tablet b7b89fcab5954a3aa1250be4a14a7bda with delete_type TABLET_DATA_DELETED (replaced by tablet b7837bc0821a4b68972b663606364955) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:23.968801 11753 raft_consensus.cc:2258] T b7b89fcab5954a3aa1250be4a14a7bda P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:23.969674 11118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_1e052cd91c654d85a0fa1da50d394b29" } modify_external_catalogs: true
W20260926 07:35:23.966619 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet acc5568dee9b473f8f5f1f8bc3b5504e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet acc5568dee9b473f8f5f1f8bc3b5504e already in progress: creating tablet
I20260926 07:35:23.970458 11118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_1e052cd91c654d85a0fa1da50d394b29" } modify_external_catalogs: true
I20260926 07:35:23.970801 11753 ts_tablet_manager.cc:1905] T b7b89fcab5954a3aa1250be4a14a7bda P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 07:35:23.973305 11117 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1620b507f07c446c8c1811895a097409 consensus state has no leader
I20260926 07:35:23.974246 11117 master_service.cc:946] ReplaceTablet: received request to replace tablet 14e5da3711fe46c4bb46a7a60f41a581 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:23.977345 11756 raft_consensus.cc:348] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.977929 11756 raft_consensus.cc:374] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:23.978142 11756 raft_consensus.cc:729] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:23.979696 11756 consensus_queue.cc:260] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:23.984479 11756 ts_tablet_manager.cc:1428] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20260926 07:35:23.986815 11443 tablet_service.cc:1514] Processing DeleteTablet for tablet acc5568dee9b473f8f5f1f8bc3b5504e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:23 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:23.988142 11756 tablet_bootstrap.cc:492] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:23.990180 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet b7b89fcab5954a3aa1250be4a14a7bda with delete_type TABLET_DATA_DELETED (replaced by tablet b7837bc0821a4b68972b663606364955) from {username='slave'} at 127.0.0.1:35244
W20260926 07:35:23.995167 11120 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e06ed286144941269b9a0c0c60bff0ce consensus state has no leader
I20260926 07:35:24.002380 11756 tablet_bootstrap.cc:654] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:24.009016 11466 raft_consensus.cc:2454] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 1.
I20260926 07:35:24.013417 11754 tablet_bootstrap.cc:492] T cb27378f5ed24e76804b05d5fb1cf441 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:24.013799 11754 ts_tablet_manager.cc:1397] T cb27378f5ed24e76804b05d5fb1cf441 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.379s user 0.015s sys 0.003s
W20260926 07:35:24.013916 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet b7b89fcab5954a3aa1250be4a14a7bda because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b7b89fcab5954a3aa1250be4a14a7bda already in progress: creating tablet
I20260926 07:35:23.993166 11116 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_89f2ea8ea48b4a38bb51ef6fb018a2e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:24.016017 11754 raft_consensus.cc:348] T cb27378f5ed24e76804b05d5fb1cf441 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:24.016983 11754 raft_consensus.cc:374] T cb27378f5ed24e76804b05d5fb1cf441 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:24.017285 11754 raft_consensus.cc:729] T cb27378f5ed24e76804b05d5fb1cf441 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
W20260926 07:35:24.018379 11116 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_89f2ea8ea48b4a38bb51ef6fb018a2e3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:24.018124 11754 consensus_queue.cc:260] T cb27378f5ed24e76804b05d5fb1cf441 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:24.019829 11755 consensus_queue.cc:1035] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:24.027740 11754 ts_tablet_manager.cc:1428] T cb27378f5ed24e76804b05d5fb1cf441 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.014s user 0.007s sys 0.000s
I20260926 07:35:24.028680 11754 tablet_bootstrap.cc:492] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:24.032940 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet cb27378f5ed24e76804b05d5fb1cf441 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:24.033648 11751 tablet_replica.cc:330] T cb27378f5ed24e76804b05d5fb1cf441 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:24.034336 11751 raft_consensus.cc:2229] T cb27378f5ed24e76804b05d5fb1cf441 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:24.034734 11751 raft_consensus.cc:2258] T cb27378f5ed24e76804b05d5fb1cf441 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:24.036720 11751 ts_tablet_manager.cc:1905] T cb27378f5ed24e76804b05d5fb1cf441 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:24.041716 11754 tablet_bootstrap.cc:654] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:24.044486 11755 consensus_queue.cc:1035] T 4ffd8bef90f14047b367a05295e976dd P 22f3902d01584936b20437606cb86dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:24.055850 11752 ts_tablet_manager.cc:1918] T 3c038c45769d4e43b15cb4d1d73a3466 P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:24.056252 11752 log.cc:1196] T 3c038c45769d4e43b15cb4d1d73a3466 P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/3c038c45769d4e43b15cb4d1d73a3466
I20260926 07:35:24.057353 11752 ts_tablet_manager.cc:1939] T 3c038c45769d4e43b15cb4d1d73a3466 P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:24.058395 11287 tablet_service.cc:1514] Processing DeleteTablet for tablet bd27f6c6ed744d7bbd0c7ab217f0b37e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:24.060662 11754 tablet_bootstrap.cc:492] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:24.061101 11754 ts_tablet_manager.cc:1397] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.033s user 0.010s sys 0.000s
I20260926 07:35:24.064493 11754 raft_consensus.cc:348] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.066172 11754 raft_consensus.cc:374] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:24.066425 11754 raft_consensus.cc:729] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:24.067160 11754 consensus_queue.cc:260] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.073781 11754 ts_tablet_manager.cc:1428] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260926 07:35:24.074960 11754 tablet_bootstrap.cc:492] T b7b89fcab5954a3aa1250be4a14a7bda P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:24.080771 11754 tablet_bootstrap.cc:654] T b7b89fcab5954a3aa1250be4a14a7bda P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:24.099718 11769 raft_consensus.cc:482] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:24.100159 11769 raft_consensus.cc:504] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.105167 11769 leader_election.cc:288] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:24.107350 11118 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet e8b2d3d86fd64704a06081b77b4e67f5
I20260926 07:35:24.107206 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a000cb9928de4459be1e6b1b7a81848a" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
W20260926 07:35:24.108700 11546 leader_election.cc:341] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:24.109242 11753 ts_tablet_manager.cc:1918] T b7b89fcab5954a3aa1250be4a14a7bda P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:24.109594 11753 log.cc:1196] T b7b89fcab5954a3aa1250be4a14a7bda P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/b7b89fcab5954a3aa1250be4a14a7bda
I20260926 07:35:24.109683 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a000cb9928de4459be1e6b1b7a81848a" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:24.110658 11753 ts_tablet_manager.cc:1939] T b7b89fcab5954a3aa1250be4a14a7bda P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:24.110810 11466 raft_consensus.cc:2454] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 0.
I20260926 07:35:24.111935 11545 leader_election.cc:302] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters: e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:24.113164 11769 raft_consensus.cc:2781] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:24.113515 11769 raft_consensus.cc:482] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:24.113867 11769 raft_consensus.cc:3037] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:24.112478 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet 3c038c45769d4e43b15cb4d1d73a3466 (table default.Table_0a660d82b80c40d8a4c41b9abdec566f [id=f23a7ed0439c41149de764ab244ef60b]) successfully deleted
W20260926 07:35:24.115183 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet aa02661e7531402687f03e86dbc19418 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa02661e7531402687f03e86dbc19418 already in progress: creating tablet
I20260926 07:35:24.115577 11287 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cc53f05536743b8a24ab2c88937f8f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 9a255ef54f7e410ca8a779e5d7a118ca) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:24.122184 11753 tablet_replica.cc:330] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:24.123199 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet b7b89fcab5954a3aa1250be4a14a7bda (table default.Table_582802febc8147cba686fb42d81b56df [id=10eb71ca30de4a7da89de44126029c68]) successfully deleted
I20260926 07:35:24.123200 11753 raft_consensus.cc:2229] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:24.124240 11753 raft_consensus.cc:2258] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:24.127369 11753 ts_tablet_manager.cc:1905] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 07:35:24.128773 11401 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.171.1:41829, user_credentials={real_user=slave}} blocked reactor thread for 53888.4us
I20260926 07:35:24.131460 11752 tablet_replica.cc:330] T cb27378f5ed24e76804b05d5fb1cf441 P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:24.132565 11752 raft_consensus.cc:2229] T cb27378f5ed24e76804b05d5fb1cf441 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:24.133492 11752 raft_consensus.cc:2258] T cb27378f5ed24e76804b05d5fb1cf441 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260926 07:35:24.133610 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 9a255ef54f7e410ca8a779e5d7a118ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9a255ef54f7e410ca8a779e5d7a118ca already in progress: creating tablet
I20260926 07:35:24.141956 11117 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 14e5da3711fe46c4bb46a7a60f41a581
I20260926 07:35:24.143265 11117 catalog_manager.cc:6565] ReplaceTablet: tablet 14e5da3711fe46c4bb46a7a60f41a581 deleted and replaced by tablet 35f8ea999baa4d38a1433975ff6ccbb5
I20260926 07:35:24.144762 11287 tablet_service.cc:1514] Processing DeleteTablet for tablet 14e5da3711fe46c4bb46a7a60f41a581 with delete_type TABLET_DATA_DELETED (replaced by tablet 35f8ea999baa4d38a1433975ff6ccbb5) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:24.152420 11756 tablet_bootstrap.cc:492] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:24.152872 11756 ts_tablet_manager.cc:1397] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.165s user 0.005s sys 0.022s
I20260926 07:35:24.154636 11586 tablet_service.cc:1514] Processing DeleteTablet for tablet 14e5da3711fe46c4bb46a7a60f41a581 with delete_type TABLET_DATA_DELETED (replaced by tablet 35f8ea999baa4d38a1433975ff6ccbb5) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:24.176205 11586 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c038c45769d4e43b15cb4d1d73a3466 with delete_type TABLET_DATA_DELETED (replaced by tablet 1620b507f07c446c8c1811895a097409) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:24.180779 11756 raft_consensus.cc:348] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.181439 11756 raft_consensus.cc:374] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:24.181675 11756 raft_consensus.cc:729] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:24.187892 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet 14e5da3711fe46c4bb46a7a60f41a581 with delete_type TABLET_DATA_DELETED (replaced by tablet 35f8ea999baa4d38a1433975ff6ccbb5) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:24.189178 11756 consensus_queue.cc:260] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.194500 11756 ts_tablet_manager.cc:1428] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.041s user 0.004s sys 0.002s
I20260926 07:35:24.195751 11756 tablet_bootstrap.cc:492] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:24.204872 11116 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_89f2ea8ea48b4a38bb51ef6fb018a2e3" } new_table_name: "default.Table_a5faf129d7cf4746a780e39333c6b5fc" modify_external_catalogs: true
I20260926 07:35:24.208498 11118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_7846bbd3f8cd4fbbb54ce45d2e3ce6b9" } modify_external_catalogs: true
I20260926 07:35:24.209012 11118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_7846bbd3f8cd4fbbb54ce45d2e3ce6b9" } modify_external_catalogs: true
I20260926 07:35:24.211018 11751 ts_tablet_manager.cc:1918] T cb27378f5ed24e76804b05d5fb1cf441 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:24.210776 11117 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_482b512cdaa04fef87180f6c8f787245" } new_table_name: "default.Table_fdc97586e1b74502987e6062bfed73d8" modify_external_catalogs: true
I20260926 07:35:24.211633 11751 log.cc:1196] T cb27378f5ed24e76804b05d5fb1cf441 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/cb27378f5ed24e76804b05d5fb1cf441
I20260926 07:35:24.211633 11769 raft_consensus.cc:504] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.212404 11751 ts_tablet_manager.cc:1939] T cb27378f5ed24e76804b05d5fb1cf441 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:24.213899 11769 leader_election.cc:288] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:24.216720 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a000cb9928de4459be1e6b1b7a81848a" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:24.218096 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a000cb9928de4459be1e6b1b7a81848a" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:24.218936 11466 raft_consensus.cc:3037] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
W20260926 07:35:24.222128 11546 leader_election.cc:341] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:24.223248 11769 raft_consensus.cc:482] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:24.227450 11113 catalog_manager.cc:5495] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e reported cstate change: term changed from 0 to 1, leader changed from <none> to e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1). New cstate: current_term: 1 leader_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } health_report { overall_health: UNKNOWN } } }
I20260926 07:35:24.228968 11113 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet 05f071b4db7847a5adbd25a71cac3ea7 (table default.Table_8b28cfa8ea704f998eae576de3ee6c92 [id=9e879069d99f41cdb271157af1a3fd1e]). Expected version 1 got 0
I20260926 07:35:24.227028 11769 raft_consensus.cc:504] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.233923 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c038c45769d4e43b15cb4d1d73a3466" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:24.234557 11315 raft_consensus.cc:2454] T 3c038c45769d4e43b15cb4d1d73a3466 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 0.
I20260926 07:35:24.235888 11546 leader_election.cc:302] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
W20260926 07:35:24.240509 11545 leader_election.cc:341] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Not found: Tablet not found: 3c038c45769d4e43b15cb4d1d73a3466
I20260926 07:35:24.242247 11769 leader_election.cc:288] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:24.242604 11119 master_service.cc:946] ReplaceTablet: received request to replace tablet 9a255ef54f7e410ca8a779e5d7a118ca from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:24.244903 11769 raft_consensus.cc:2781] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:24.245222 11769 raft_consensus.cc:482] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:24.246762 11769 raft_consensus.cc:3037] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:24.248524 11586 tablet_service.cc:1467] Processing CreateTablet for tablet 1e1cea4523a64428aac03ca0bdf708c7 (DEFAULT_TABLE table=default.Table_7846bbd3f8cd4fbbb54ce45d2e3ce6b9 [id=41849849f4ae4a9396ccda32d995e21a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:24.249568 11586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e1cea4523a64428aac03ca0bdf708c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:24.252905 11752 ts_tablet_manager.cc:1905] T cb27378f5ed24e76804b05d5fb1cf441 P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:24.258109 11746 tablet_bootstrap.cc:492] T b7b89fcab5954a3aa1250be4a14a7bda P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:24.258536 11746 ts_tablet_manager.cc:1397] T b7b89fcab5954a3aa1250be4a14a7bda P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.460s user 0.017s sys 0.013s
I20260926 07:35:24.251730 11756 tablet_bootstrap.cc:654] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:24.267555 11287 tablet_service.cc:1467] Processing CreateTablet for tablet 1e1cea4523a64428aac03ca0bdf708c7 (DEFAULT_TABLE table=default.Table_7846bbd3f8cd4fbbb54ce45d2e3ce6b9 [id=41849849f4ae4a9396ccda32d995e21a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:24.268672 11287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e1cea4523a64428aac03ca0bdf708c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:24.268723 11442 tablet_service.cc:1467] Processing CreateTablet for tablet a55e54d492214039b0294f954b5ef390 (DEFAULT_TABLE table=default.Table_b413c386b9e346668b23e2ad025e2b7b [id=65c112d56f2a44cc9d840caadd218412]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:24.269927 11442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a55e54d492214039b0294f954b5ef390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:24.269989 11585 tablet_service.cc:1467] Processing CreateTablet for tablet a55e54d492214039b0294f954b5ef390 (DEFAULT_TABLE table=default.Table_b413c386b9e346668b23e2ad025e2b7b [id=65c112d56f2a44cc9d840caadd218412]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:24.271037 11585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a55e54d492214039b0294f954b5ef390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:24.274296 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet cb27378f5ed24e76804b05d5fb1cf441 (table default.Table_cb7f69c004f64db7bdc4412924ca513e [id=4c1477cba48a45558d6e0dfe08cd8afa]) successfully deleted
I20260926 07:35:24.275290 11115 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_c961864a78e341a9bf757eaebadc4e84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:24.276787 11441 tablet_service.cc:1467] Processing CreateTablet for tablet 1e1cea4523a64428aac03ca0bdf708c7 (DEFAULT_TABLE table=default.Table_7846bbd3f8cd4fbbb54ce45d2e3ce6b9 [id=41849849f4ae4a9396ccda32d995e21a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:24.278270 11441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e1cea4523a64428aac03ca0bdf708c7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:24.279980 11116 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 55ad1d96e2314d1eb0a717a3e9ff31bc has no consensus state
I20260926 07:35:24.279980 11288 tablet_service.cc:1467] Processing CreateTablet for tablet a55e54d492214039b0294f954b5ef390 (DEFAULT_TABLE table=default.Table_b413c386b9e346668b23e2ad025e2b7b [id=65c112d56f2a44cc9d840caadd218412]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:24.281108 11288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a55e54d492214039b0294f954b5ef390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:24.283156 11443 tablet_service.cc:1467] Processing CreateTablet for tablet 509f607b089c44bc9c3f80ed59a68c95 (DEFAULT_TABLE table=default.Table_65217d931d594a70b45c7d5c486930ba [id=b0949f4718f8482881854a65f89d7421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:24.284159 11443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 509f607b089c44bc9c3f80ed59a68c95. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:24.284981 11115 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c961864a78e341a9bf757eaebadc4e84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:24.296777 11754 tablet_bootstrap.cc:492] T b7b89fcab5954a3aa1250be4a14a7bda P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:24.297202 11754 ts_tablet_manager.cc:1397] T b7b89fcab5954a3aa1250be4a14a7bda P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.222s user 0.014s sys 0.015s
W20260926 07:35:24.297930 11117 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ae16af9edff9401589bcf54eeb3e8dfe has no consensus state
I20260926 07:35:24.299348 11754 raft_consensus.cc:348] T b7b89fcab5954a3aa1250be4a14a7bda P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
W20260926 07:35:24.301254 11546 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.1:41829, user_credentials={real_user=slave}} blocked reactor thread for 65419.9us
I20260926 07:35:24.304884 11583 tablet_service.cc:1467] Processing CreateTablet for tablet 509f607b089c44bc9c3f80ed59a68c95 (DEFAULT_TABLE table=default.Table_65217d931d594a70b45c7d5c486930ba [id=b0949f4718f8482881854a65f89d7421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:24.306170 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 509f607b089c44bc9c3f80ed59a68c95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:24.312011 11111 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_ba9d6249f2594f7687befc65703d5f25"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:24.313795 11111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba9d6249f2594f7687befc65703d5f25 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:24.316275 11756 tablet_bootstrap.cc:492] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:24.316689 11756 ts_tablet_manager.cc:1397] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.121s user 0.010s sys 0.000s
I20260926 07:35:24.318802 11756 raft_consensus.cc:348] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:24.319435 11756 raft_consensus.cc:374] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:24.319700 11756 raft_consensus.cc:729] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:24.321524 11466 raft_consensus.cc:2454] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 1.
I20260926 07:35:24.299991 11754 raft_consensus.cc:374] T b7b89fcab5954a3aa1250be4a14a7bda P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:24.325480 11754 raft_consensus.cc:729] T b7b89fcab5954a3aa1250be4a14a7bda P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:24.326187 11754 consensus_queue.cc:260] T b7b89fcab5954a3aa1250be4a14a7bda P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
W20260926 07:35:24.332635 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 14e5da3711fe46c4bb46a7a60f41a581 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14e5da3711fe46c4bb46a7a60f41a581 already in progress: creating tablet
I20260926 07:35:24.333359 11119 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_783e2f568d774b5eb908860af5fa4166"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:24.334573 11754 ts_tablet_manager.cc:1428] T b7b89fcab5954a3aa1250be4a14a7bda P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.037s user 0.006s sys 0.000s
I20260926 07:35:24.335215 11756 consensus_queue.cc:260] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:24.339314 11545 leader_election.cc:302] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters: e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:24.341018 11287 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c038c45769d4e43b15cb4d1d73a3466 with delete_type TABLET_DATA_DELETED (replaced by tablet 1620b507f07c446c8c1811895a097409) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:24.341666 11292 tablet_service.cc:1514] Processing DeleteTablet for tablet 7000bcd55a984d64ba9d7d50fe63822c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:24.342533 11751 tablet_replica.cc:330] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:24.344182 11584 tablet_service.cc:1514] Processing DeleteTablet for tablet b7b89fcab5954a3aa1250be4a14a7bda with delete_type TABLET_DATA_DELETED (replaced by tablet b7837bc0821a4b68972b663606364955) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:24.343075 11820 raft_consensus.cc:2781] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:24.344836 11754 tablet_bootstrap.cc:492] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
W20260926 07:35:24.347720 11119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_783e2f568d774b5eb908860af5fa4166 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:24.352445 11756 ts_tablet_manager.cc:1428] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.036s user 0.006s sys 0.000s
I20260926 07:35:24.353587 11756 tablet_bootstrap.cc:492] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:24.356344 11753 ts_tablet_manager.cc:1918] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:24.356760 11753 log.cc:1196] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/bd27f6c6ed744d7bbd0c7ab217f0b37e
I20260926 07:35:24.357481 11753 ts_tablet_manager.cc:1939] T bd27f6c6ed744d7bbd0c7ab217f0b37e P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:24.365984 11118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 1e1cea4523a64428aac03ca0bdf708c7
I20260926 07:35:24.367770 11769 raft_consensus.cc:504] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.370615 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c038c45769d4e43b15cb4d1d73a3466" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:24.371066 11445 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e1cea4523a64428aac03ca0bdf708c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:24 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:24.371119 11315 raft_consensus.cc:3037] T 3c038c45769d4e43b15cb4d1d73a3466 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:24.371716 11584 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e1cea4523a64428aac03ca0bdf708c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:24 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:24.373628 11113 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_ff479f29ec0143d6a74b38e25389f1bc" } modify_external_catalogs: true
I20260926 07:35:24.374146 11113 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_ff479f29ec0143d6a74b38e25389f1bc" } modify_external_catalogs: true
I20260926 07:35:24.378170 11751 raft_consensus.cc:2229] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus shutting down.
W20260926 07:35:24.388532 11545 leader_election.cc:341] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Not found: Tablet not found: 3c038c45769d4e43b15cb4d1d73a3466
I20260926 07:35:24.389096 11769 leader_election.cc:288] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:24.393178 11746 raft_consensus.cc:348] T b7b89fcab5954a3aa1250be4a14a7bda P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.394027 11746 raft_consensus.cc:374] T b7b89fcab5954a3aa1250be4a14a7bda P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:24.394416 11746 raft_consensus.cc:729] T b7b89fcab5954a3aa1250be4a14a7bda P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:24.395279 11746 consensus_queue.cc:260] T b7b89fcab5954a3aa1250be4a14a7bda P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.403607 11118 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_904e49ad16464cdbac83d37d3cdf115b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:24.405380 11803 raft_consensus.cc:482] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260926 07:35:24.405954 11118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_904e49ad16464cdbac83d37d3cdf115b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:24.405804 11803 raft_consensus.cc:504] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:24.407793 11803 leader_election.cc:288] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
W20260926 07:35:24.410216 11249 leader_election.cc:341] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Not found: Tablet not found: cb27378f5ed24e76804b05d5fb1cf441
I20260926 07:35:24.411852 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb27378f5ed24e76804b05d5fb1cf441" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
W20260926 07:35:24.412940 11247 leader_election.cc:341] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: Tablet not RUNNING: STOPPED
I20260926 07:35:24.413332 11247 leader_election.cc:302] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade
I20260926 07:35:24.414407 11803 raft_consensus.cc:2726] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:24.414927 11753 tablet_replica.cc:330] T 8cc53f05536743b8a24ab2c88937f8f2 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:24.415056 11315 raft_consensus.cc:2454] T 3c038c45769d4e43b15cb4d1d73a3466 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 1.
I20260926 07:35:24.416757 11546 leader_election.cc:302] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1
I20260926 07:35:24.420334 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet bd27f6c6ed744d7bbd0c7ab217f0b37e (table default.Table_6ada159fbff8487c85708620316caa04 [id=f232e32225aa4d1a82f4f061434e9f89]) successfully deleted
I20260926 07:35:24.421588 11292 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e1cea4523a64428aac03ca0bdf708c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:24 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:24.422614 11292 tablet_service.cc:1514] Processing DeleteTablet for tablet 50d0e4cd192c445bb7c82d1a3c4a53d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:24.415962 11753 raft_consensus.cc:2229] T 8cc53f05536743b8a24ab2c88937f8f2 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:24.425311 11753 raft_consensus.cc:2258] T 8cc53f05536743b8a24ab2c88937f8f2 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:24.426580 11769 raft_consensus.cc:2684] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Received election callback for term 1 while not running: Illegal state: RaftConsensus is not running: State = Stopping
I20260926 07:35:24.427179 11292 tablet_service.cc:1514] Processing DeleteTablet for tablet cb27378f5ed24e76804b05d5fb1cf441 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:24.429253 11751 raft_consensus.cc:2258] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:24.427196 11753 ts_tablet_manager.cc:1905] T 8cc53f05536743b8a24ab2c88937f8f2 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:24.446139 11754 tablet_bootstrap.cc:654] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:24.455883 11756 tablet_bootstrap.cc:654] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:24.456027 11292 tablet_service.cc:1467] Processing CreateTablet for tablet 509f607b089c44bc9c3f80ed59a68c95 (DEFAULT_TABLE table=default.Table_65217d931d594a70b45c7d5c486930ba [id=b0949f4718f8482881854a65f89d7421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:24.462635 11751 ts_tablet_manager.cc:1905] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:24.462613 11116 master_service.cc:946] ReplaceTablet: received request to replace tablet 3c038c45769d4e43b15cb4d1d73a3466 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:24.466053 11820 raft_consensus.cc:686] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Becoming Leader. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Running, Role: LEADER
I20260926 07:35:24.467183 11820 consensus_queue.cc:237] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.468698 11746 ts_tablet_manager.cc:1428] T b7b89fcab5954a3aa1250be4a14a7bda P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.210s user 0.008s sys 0.000s
I20260926 07:35:24.469822 11746 tablet_bootstrap.cc:492] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:24.485589 11117 master_service.cc:946] ReplaceTablet: received request to replace tablet acc5568dee9b473f8f5f1f8bc3b5504e from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:24.498428 11756 tablet_bootstrap.cc:492] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:24.500805 11746 tablet_bootstrap.cc:654] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:24.503883 11114 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_a5faf129d7cf4746a780e39333c6b5fc" } new_table_name: "default.Table_4d463023afd047528c16b7ee4dabd36f" modify_external_catalogs: true
I20260926 07:35:24.505750 11115 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_e2ca52cf31c34e1cb2e1cf4bd05510c0" } new_table_name: "default.Table_eb3d1f18577c4064b04bceab813f25e6" modify_external_catalogs: true
I20260926 07:35:24.518106 11292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 509f607b089c44bc9c3f80ed59a68c95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:24.518368 11610 raft_consensus.cc:1261] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Refusing update from remote peer e9bfebb34fa14cd2ad4843fffbe5c54e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:35:24.520001 11466 raft_consensus.cc:1261] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Refusing update from remote peer e9bfebb34fa14cd2ad4843fffbe5c54e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:35:24.520860 11803 consensus_queue.cc:1035] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 07:35:24.522830 11803 consensus_queue.cc:1035] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:24.539475 11756 ts_tablet_manager.cc:1397] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.186s user 0.008s sys 0.005s
I20260926 07:35:24.554800 11754 tablet_bootstrap.cc:492] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:24.565984 11756 raft_consensus.cc:348] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.567149 11756 raft_consensus.cc:374] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:24.567396 11756 raft_consensus.cc:729] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:24.567875 11816 mvcc.cc:205] Tried to move back new op lower bound from 7333511677978832896 to 7333511675538690048. Current Snapshot: MvccSnapshot[applied={T|T < 7333511677978832896}]
I20260926 07:35:24.555296 11754 ts_tablet_manager.cc:1397] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.211s user 0.010s sys 0.001s
I20260926 07:35:24.578029 11119 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_22c5272f755c4a659ac71d7a5e1c3a22" } modify_external_catalogs: true
I20260926 07:35:24.578852 11119 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_22c5272f755c4a659ac71d7a5e1c3a22" } modify_external_catalogs: true
I20260926 07:35:24.579421 11831 tablet.cc:1722] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 07:35:24.582631 11756 consensus_queue.cc:260] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.585464 11756 ts_tablet_manager.cc:1428] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.046s user 0.003s sys 0.002s
I20260926 07:35:24.571611 11754 raft_consensus.cc:348] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.590080 11754 raft_consensus.cc:374] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:24.607293 11819 mvcc.cc:205] Tried to move back new op lower bound from 7333511677978832896 to 7333511675538690048. Current Snapshot: MvccSnapshot[applied={T|T < 7333511677978832896}]
I20260926 07:35:24.590408 11754 raft_consensus.cc:729] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:24.613988 11753 ts_tablet_manager.cc:1918] T 8cc53f05536743b8a24ab2c88937f8f2 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:24.614532 11753 log.cc:1196] T 8cc53f05536743b8a24ab2c88937f8f2 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/8cc53f05536743b8a24ab2c88937f8f2
I20260926 07:35:24.615248 11753 ts_tablet_manager.cc:1939] T 8cc53f05536743b8a24ab2c88937f8f2 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:24.615161 11751 ts_tablet_manager.cc:1918] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:24.615528 11751 log.cc:1196] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/3c038c45769d4e43b15cb4d1d73a3466
I20260926 07:35:24.616183 11751 ts_tablet_manager.cc:1939] T 3c038c45769d4e43b15cb4d1d73a3466 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:24.618332 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet 8cc53f05536743b8a24ab2c88937f8f2 (table default.Table_836a28372ec4439d98b1583a9b50d2c1 [id=800a96cbd0b34639bbe7ad317a640053]) successfully deleted
I20260926 07:35:24.619128 11113 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 35f8ea999baa4d38a1433975ff6ccbb5
I20260926 07:35:24.607919 11756 tablet_bootstrap.cc:492] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:24.608538 11754 consensus_queue.cc:260] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.623376 11754 ts_tablet_manager.cc:1428] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.055s user 0.005s sys 0.000s
I20260926 07:35:24.623198 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_1369e71c66c446029e44aca42bf1b700"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:24.625559 11753 tablet_replica.cc:330] T 3c038c45769d4e43b15cb4d1d73a3466 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:24.626284 11753 raft_consensus.cc:2229] T 3c038c45769d4e43b15cb4d1d73a3466 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:24.626703 11753 raft_consensus.cc:2258] T 3c038c45769d4e43b15cb4d1d73a3466 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:24.639117 11751 tablet_replica.cc:330] T b7b89fcab5954a3aa1250be4a14a7bda P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:24.638914 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 3c038c45769d4e43b15cb4d1d73a3466 (table default.Table_0a660d82b80c40d8a4c41b9abdec566f [id=f23a7ed0439c41149de764ab244ef60b]) successfully deleted
I20260926 07:35:24.639796 11751 raft_consensus.cc:2229] T b7b89fcab5954a3aa1250be4a14a7bda P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:24.640300 11751 raft_consensus.cc:2258] T b7b89fcab5954a3aa1250be4a14a7bda P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:24.642417 11751 ts_tablet_manager.cc:1905] T b7b89fcab5954a3aa1250be4a14a7bda P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:24.644423 11117 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet a000cb9928de4459be1e6b1b7a81848a (table default.Table_d0380e9d3f26439bbc91cf476aeae69f [id=19875953f2b6484f934503404af57593]). Expected version 2 got 0
I20260926 07:35:24.647039 11118 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 05f071b4db7847a5adbd25a71cac3ea7 (table default.Table_8b28cfa8ea704f998eae576de3ee6c92 [id=9e879069d99f41cdb271157af1a3fd1e]). Expected version 1 got 0
I20260926 07:35:24.647703 11754 tablet_bootstrap.cc:492] T 8cc53f05536743b8a24ab2c88937f8f2 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
W20260926 07:35:24.652698 11113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1369e71c66c446029e44aca42bf1b700 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:35:24.654898 11249 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.171.3:45127, user_credentials={real_user=slave}} blocked reactor thread for 70484.5us
I20260926 07:35:24.658152 11756 tablet_bootstrap.cc:654] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:24.658223 11835 tablet.cc:1722] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 07:35:24.666294 11828 raft_consensus.cc:482] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:24.666968 11118 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 05f071b4db7847a5adbd25a71cac3ea7 (table default.Table_8b28cfa8ea704f998eae576de3ee6c92 [id=9e879069d99f41cdb271157af1a3fd1e]). Expected version 1 got 0
I20260926 07:35:24.666719 11828 raft_consensus.cc:504] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.668840 11828 leader_election.cc:288] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:24.670701 11832 tablet.cc:1722] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 07:35:24.673156 11611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7000bcd55a984d64ba9d7d50fe63822c" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
W20260926 07:35:24.674619 11249 leader_election.cc:341] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:24.675416 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7000bcd55a984d64ba9d7d50fe63822c" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
W20260926 07:35:24.676601 11247 leader_election.cc:341] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:24.676964 11247 leader_election.cc:302] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade
I20260926 07:35:24.677857 11803 raft_consensus.cc:2726] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:24.679013 11753 ts_tablet_manager.cc:1905] T 3c038c45769d4e43b15cb4d1d73a3466 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:24.690973 11116 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet a000cb9928de4459be1e6b1b7a81848a (table default.Table_d0380e9d3f26439bbc91cf476aeae69f [id=19875953f2b6484f934503404af57593]). Expected version 2 got 0
I20260926 07:35:24.691805 11754 tablet_bootstrap.cc:654] T 8cc53f05536743b8a24ab2c88937f8f2 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:24.706604 11746 tablet_bootstrap.cc:492] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:24.707226 11746 ts_tablet_manager.cc:1397] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.238s user 0.008s sys 0.006s
I20260926 07:35:24.709198 11746 raft_consensus.cc:348] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.709968 11746 raft_consensus.cc:374] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:24.710244 11746 raft_consensus.cc:729] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:24.710871 11746 consensus_queue.cc:260] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.720531 11756 tablet_bootstrap.cc:492] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:24.720898 11756 ts_tablet_manager.cc:1397] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.127s user 0.008s sys 0.001s
I20260926 07:35:24.722978 11756 raft_consensus.cc:348] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.723594 11756 raft_consensus.cc:374] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:24.723820 11756 raft_consensus.cc:729] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:24.724716 11756 consensus_queue.cc:260] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.726589 11756 ts_tablet_manager.cc:1428] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260926 07:35:24.727331 11756 tablet_bootstrap.cc:492] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:24.744143 11756 tablet_bootstrap.cc:654] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:24.715981 11752 ts_tablet_manager.cc:1918] T cb27378f5ed24e76804b05d5fb1cf441 P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:24.751142 11752 log.cc:1196] T cb27378f5ed24e76804b05d5fb1cf441 P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/cb27378f5ed24e76804b05d5fb1cf441
I20260926 07:35:24.751798 11752 ts_tablet_manager.cc:1939] T cb27378f5ed24e76804b05d5fb1cf441 P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:24.758096 11746 ts_tablet_manager.cc:1428] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.051s user 0.005s sys 0.000s
I20260926 07:35:24.762589 11756 tablet_bootstrap.cc:492] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:24.762955 11756 ts_tablet_manager.cc:1397] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.036s user 0.012s sys 0.000s
I20260926 07:35:24.766752 11754 tablet_bootstrap.cc:492] T 8cc53f05536743b8a24ab2c88937f8f2 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:24.767100 11754 ts_tablet_manager.cc:1397] T 8cc53f05536743b8a24ab2c88937f8f2 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.120s user 0.007s sys 0.001s
I20260926 07:35:24.694990 11117 master_service.cc:946] ReplaceTablet: received request to replace tablet cb27378f5ed24e76804b05d5fb1cf441 from {username='slave'} at 127.0.0.1:44836
W20260926 07:35:24.729537 11114 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 55ad1d96e2314d1eb0a717a3e9ff31bc has no consensus state
W20260926 07:35:24.730429 11115 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f31a069dc888487ebbbb1cb91e886ad0 consensus state has no leader
I20260926 07:35:24.768268 11756 raft_consensus.cc:348] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.768905 11756 raft_consensus.cc:374] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:24.769138 11756 raft_consensus.cc:729] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:24.769793 11754 raft_consensus.cc:348] T 8cc53f05536743b8a24ab2c88937f8f2 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.770417 11754 raft_consensus.cc:374] T 8cc53f05536743b8a24ab2c88937f8f2 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:24.770632 11754 raft_consensus.cc:729] T 8cc53f05536743b8a24ab2c88937f8f2 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:24.771049 11756 consensus_queue.cc:260] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.771126 11754 consensus_queue.cc:260] T 8cc53f05536743b8a24ab2c88937f8f2 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.771674 11111 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_b9f92e070aab45cdb8352c1d8a3d603c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:24.773411 11111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b9f92e070aab45cdb8352c1d8a3d603c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:24.775590 11756 ts_tablet_manager.cc:1428] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.011s user 0.004s sys 0.000s
I20260926 07:35:24.776373 11756 tablet_bootstrap.cc:492] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:24.787341 11115 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_03b976b95c404da1b2b14a6d71dd8a72"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:24.789485 11443 tablet_service.cc:1514] Processing DeleteTablet for tablet 50d0e4cd192c445bb7c82d1a3c4a53d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:40244
W20260926 07:35:24.790083 11115 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_03b976b95c404da1b2b14a6d71dd8a72 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:24.792477 11120 master_service.cc:946] ReplaceTablet: received request to replace tablet 345d14fc9eea46ad9b1155d538118ee5 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:24.797254 11746 tablet_bootstrap.cc:492] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:24.802088 11746 tablet_bootstrap.cc:654] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:24.803582 11756 tablet_bootstrap.cc:654] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:24.759999 11119 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet e06ed286144941269b9a0c0c60bff0ce
W20260926 07:35:24.806918 11104 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 55ad1d96e2314d1eb0a717a3e9ff31bc (table default.Table_4d463023afd047528c16b7ee4dabd36f [id=eeb221378f35425eb8d24b815f8a2181], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 55ad1d96e2314d1eb0a717a3e9ff31bc has no consensus state
I20260926 07:35:24.807591 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet e06ed286144941269b9a0c0c60bff0ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:24 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:24.809151 11443 tablet_service.cc:1514] Processing DeleteTablet for tablet e06ed286144941269b9a0c0c60bff0ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:24 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:24.817795 11753 ts_tablet_manager.cc:1918] T 3c038c45769d4e43b15cb4d1d73a3466 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:24.818214 11753 log.cc:1196] T 3c038c45769d4e43b15cb4d1d73a3466 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/3c038c45769d4e43b15cb4d1d73a3466
I20260926 07:35:24.818863 11753 ts_tablet_manager.cc:1939] T 3c038c45769d4e43b15cb4d1d73a3466 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:24.821971 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet 3c038c45769d4e43b15cb4d1d73a3466 (table default.Table_0a660d82b80c40d8a4c41b9abdec566f [id=f23a7ed0439c41149de764ab244ef60b]) successfully deleted
I20260926 07:35:24.824105 11756 tablet_bootstrap.cc:492] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:24.824499 11756 ts_tablet_manager.cc:1397] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.048s user 0.010s sys 0.000s
I20260926 07:35:24.824611 11753 tablet_replica.cc:330] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:24.825258 11753 raft_consensus.cc:2229] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:24.825636 11753 raft_consensus.cc:2258] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:24.826850 11752 tablet_replica.cc:330] T b7b89fcab5954a3aa1250be4a14a7bda P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:24.827394 11753 ts_tablet_manager.cc:1905] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:24.827571 11752 raft_consensus.cc:2229] T b7b89fcab5954a3aa1250be4a14a7bda P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:24.827184 11120 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_2f42e2e9196745bb942de35a1373e040"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:24.827987 11752 raft_consensus.cc:2258] T b7b89fcab5954a3aa1250be4a14a7bda P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260926 07:35:24.829507 11120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2f42e2e9196745bb942de35a1373e040 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:24.830250 11752 ts_tablet_manager.cc:1905] T b7b89fcab5954a3aa1250be4a14a7bda P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:24.832588 11114 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_b5e830fcbaa44c209d50a84038d3b68f" } modify_external_catalogs: true
I20260926 07:35:24.833403 11114 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_b5e830fcbaa44c209d50a84038d3b68f" } modify_external_catalogs: true
I20260926 07:35:24.836702 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet cb27378f5ed24e76804b05d5fb1cf441 (table default.Table_cb7f69c004f64db7bdc4412924ca513e [id=4c1477cba48a45558d6e0dfe08cd8afa]) successfully deleted
I20260926 07:35:24.842998 11756 raft_consensus.cc:348] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.843622 11756 raft_consensus.cc:374] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:24.843858 11756 raft_consensus.cc:729] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:24.844511 11756 consensus_queue.cc:260] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.846131 11754 ts_tablet_manager.cc:1428] T 8cc53f05536743b8a24ab2c88937f8f2 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.079s user 0.002s sys 0.005s
I20260926 07:35:24.855295 11756 ts_tablet_manager.cc:1428] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.031s user 0.005s sys 0.000s
I20260926 07:35:24.856832 11756 tablet_bootstrap.cc:492] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:24.860517 11754 tablet_bootstrap.cc:492] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:24.866406 11756 tablet_bootstrap.cc:654] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:24.882284 11833 raft_consensus.cc:482] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:24.882987 11833 raft_consensus.cc:504] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.884693 11814 raft_consensus.cc:482] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:24.885346 11833 leader_election.cc:288] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:24.885205 11814 raft_consensus.cc:504] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.886190 11611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a000cb9928de4459be1e6b1b7a81848a" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:24.887006 11814 leader_election.cc:288] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:24.888717 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50d0e4cd192c445bb7c82d1a3c4a53d1" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:24.889321 11315 raft_consensus.cc:2454] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:24.889803 11611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50d0e4cd192c445bb7c82d1a3c4a53d1" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:24.890180 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a000cb9928de4459be1e6b1b7a81848a" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:24.890478 11611 raft_consensus.cc:2454] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:24.890825 11466 raft_consensus.cc:2379] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9bfebb34fa14cd2ad4843fffbe5c54e in current term 1: Already voted for candidate 6070e9e3d1234a9d83e49431416afade in this term.
I20260926 07:35:24.892561 11401 leader_election.cc:302] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:24.893036 11746 tablet_bootstrap.cc:492] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:24.893463 11746 ts_tablet_manager.cc:1397] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.097s user 0.007s sys 0.002s
I20260926 07:35:24.894119 11247 leader_election.cc:302] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade
I20260926 07:35:24.894831 11814 raft_consensus.cc:2781] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:24.895089 11833 raft_consensus.cc:3037] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:24.895170 11814 raft_consensus.cc:482] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:24.895545 11814 raft_consensus.cc:3037] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:24.895727 11746 raft_consensus.cc:348] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.896409 11746 raft_consensus.cc:374] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:24.896628 11746 raft_consensus.cc:729] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:24.897603 11746 consensus_queue.cc:260] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.899273 11754 tablet_bootstrap.cc:654] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:24.907500 11746 ts_tablet_manager.cc:1428] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.014s user 0.003s sys 0.005s
I20260926 07:35:24.908324 11746 tablet_bootstrap.cc:492] T 7000bcd55a984d64ba9d7d50fe63822c P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:24.913030 11746 tablet_bootstrap.cc:654] T 7000bcd55a984d64ba9d7d50fe63822c P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:24.918286 11751 ts_tablet_manager.cc:1918] T b7b89fcab5954a3aa1250be4a14a7bda P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:24.918610 11751 log.cc:1196] T b7b89fcab5954a3aa1250be4a14a7bda P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/b7b89fcab5954a3aa1250be4a14a7bda
I20260926 07:35:24.919165 11751 ts_tablet_manager.cc:1939] T b7b89fcab5954a3aa1250be4a14a7bda P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:24.920401 11814 raft_consensus.cc:504] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.922132 11814 leader_election.cc:288] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:24.923306 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50d0e4cd192c445bb7c82d1a3c4a53d1" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:24.923815 11315 raft_consensus.cc:3037] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:24.924410 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet b7b89fcab5954a3aa1250be4a14a7bda (table default.Table_582802febc8147cba686fb42d81b56df [id=10eb71ca30de4a7da89de44126029c68]) successfully deleted
W20260926 07:35:24.925240 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 1e1cea4523a64428aac03ca0bdf708c7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1e1cea4523a64428aac03ca0bdf708c7 already in progress: creating tablet
I20260926 07:35:24.925284 11611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50d0e4cd192c445bb7c82d1a3c4a53d1" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:24.925769 11611 raft_consensus.cc:3037] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:24.926441 11292 tablet_service.cc:1514] Processing DeleteTablet for tablet e06ed286144941269b9a0c0c60bff0ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:24 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:24.927103 11114 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet afa901cdc6a84220bf36ee669a955b72
I20260926 07:35:24.928798 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet afa901cdc6a84220bf36ee669a955b72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:24 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:24.930495 11814 raft_consensus.cc:482] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:24.930883 11814 raft_consensus.cc:504] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.931957 11746 tablet_bootstrap.cc:492] T 7000bcd55a984d64ba9d7d50fe63822c P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:24.932324 11746 ts_tablet_manager.cc:1397] T 7000bcd55a984d64ba9d7d50fe63822c P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.003s
W20260926 07:35:24.932823 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet e06ed286144941269b9a0c0c60bff0ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e06ed286144941269b9a0c0c60bff0ce already in progress: creating tablet
I20260926 07:35:24.933754 11833 raft_consensus.cc:2726] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:24.933754 11610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cc53f05536743b8a24ab2c88937f8f2" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
W20260926 07:35:24.933998 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet afa901cdc6a84220bf36ee669a955b72 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet afa901cdc6a84220bf36ee669a955b72 already in progress: creating tablet
I20260926 07:35:24.934309 11610 raft_consensus.cc:2454] T 8cc53f05536743b8a24ab2c88937f8f2 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:24.937098 11746 raft_consensus.cc:348] T 7000bcd55a984d64ba9d7d50fe63822c P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.937695 11746 raft_consensus.cc:374] T 7000bcd55a984d64ba9d7d50fe63822c P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:24.937937 11746 raft_consensus.cc:729] T 7000bcd55a984d64ba9d7d50fe63822c P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:24.938683 11402 leader_election.cc:302] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:24.939634 11842 raft_consensus.cc:2781] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:24.940088 11842 raft_consensus.cc:482] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:24.940088 11746 consensus_queue.cc:260] T 7000bcd55a984d64ba9d7d50fe63822c P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.940589 11842 raft_consensus.cc:3037] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:24.952543 11292 tablet_service.cc:1514] Processing DeleteTablet for tablet afa901cdc6a84220bf36ee669a955b72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:24 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:24.956138 11814 leader_election.cc:288] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:24.959419 11754 tablet_bootstrap.cc:492] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:24.960023 11754 ts_tablet_manager.cc:1397] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.100s user 0.008s sys 0.004s
W20260926 07:35:24.959954 11401 leader_election.cc:341] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Not found: Tablet not found: 8cc53f05536743b8a24ab2c88937f8f2
I20260926 07:35:24.962384 11754 raft_consensus.cc:348] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.964723 11754 raft_consensus.cc:374] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:24.965104 11754 raft_consensus.cc:729] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:24.966724 11754 consensus_queue.cc:260] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:24.970095 11754 ts_tablet_manager.cc:1428] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260926 07:35:24.973022 11746 ts_tablet_manager.cc:1428] T 7000bcd55a984d64ba9d7d50fe63822c P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.040s user 0.001s sys 0.008s
I20260926 07:35:24.973866 11746 tablet_bootstrap.cc:492] T 345d14fc9eea46ad9b1155d538118ee5 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:24.976562 11754 tablet_bootstrap.cc:492] T aa02661e7531402687f03e86dbc19418 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:24.981108 11746 tablet_bootstrap.cc:654] T 345d14fc9eea46ad9b1155d538118ee5 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:24.981350 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_4fc371a1e3de4170846c211c5d8ed72f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:24.983877 11119 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_5975ffa7405849b0858063988326912b" } new_table_name: "default.Table_c7fb27daeba04d90bc0cf0c7ac07fe56" modify_external_catalogs: true
I20260926 07:35:24.992503 11111 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_1369e71c66c446029e44aca42bf1b700" } new_table_name: "default.Table_686ba040299440d0af09a9fe3501a040" modify_external_catalogs: true
I20260926 07:35:24.993562 11754 tablet_bootstrap.cc:654] T aa02661e7531402687f03e86dbc19418 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:24.994952 11746 tablet_bootstrap.cc:492] T 345d14fc9eea46ad9b1155d538118ee5 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:24.995340 11746 ts_tablet_manager.cc:1397] T 345d14fc9eea46ad9b1155d538118ee5 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.000s
I20260926 07:35:25.008540 11746 raft_consensus.cc:348] T 345d14fc9eea46ad9b1155d538118ee5 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.009214 11746 raft_consensus.cc:374] T 345d14fc9eea46ad9b1155d538118ee5 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:25.009846 11746 raft_consensus.cc:729] T 345d14fc9eea46ad9b1155d538118ee5 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:25.014122 11746 consensus_queue.cc:260] T 345d14fc9eea46ad9b1155d538118ee5 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.020210 11753 ts_tablet_manager.cc:1918] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:25.020681 11753 log.cc:1196] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/7000bcd55a984d64ba9d7d50fe63822c
I20260926 07:35:25.021386 11753 ts_tablet_manager.cc:1939] T 7000bcd55a984d64ba9d7d50fe63822c P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:25.021986 11315 raft_consensus.cc:2454] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:25.023130 11611 raft_consensus.cc:2454] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:25.033718 11753 tablet_replica.cc:330] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:25.034538 11753 raft_consensus.cc:2229] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:25.035089 11753 raft_consensus.cc:2258] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:25.038379 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet 7000bcd55a984d64ba9d7d50fe63822c (table default.Table_4bc696441dce4bc687a0344b93eccb9e [id=12a935394c1f49158f105705ce038532]) successfully deleted
W20260926 07:35:25.039209 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 1e1cea4523a64428aac03ca0bdf708c7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1e1cea4523a64428aac03ca0bdf708c7 already in progress: creating tablet
I20260926 07:35:25.041936 11753 ts_tablet_manager.cc:1905] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:25.046046 11112 master_service.cc:946] ReplaceTablet: received request to replace tablet 067ae4ea43f54e88874d93b4393dae27 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:25.054487 11120 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_de31b81bd131450d913e95011f68aa2d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:25.055864 11754 tablet_bootstrap.cc:492] T aa02661e7531402687f03e86dbc19418 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:25.056360 11754 ts_tablet_manager.cc:1397] T aa02661e7531402687f03e86dbc19418 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.085s user 0.012s sys 0.004s
I20260926 07:35:25.019591 11746 ts_tablet_manager.cc:1428] T 345d14fc9eea46ad9b1155d538118ee5 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.024s user 0.006s sys 0.001s
I20260926 07:35:25.020105 11752 ts_tablet_manager.cc:1918] T b7b89fcab5954a3aa1250be4a14a7bda P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:25.059906 11752 log.cc:1196] T b7b89fcab5954a3aa1250be4a14a7bda P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/b7b89fcab5954a3aa1250be4a14a7bda
I20260926 07:35:25.060976 11752 ts_tablet_manager.cc:1939] T b7b89fcab5954a3aa1250be4a14a7bda P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:25.061898 11116 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: default.Table_8b28cfa8ea704f998eae576de3ee6c92 [id=9e879069d99f41cdb271157af1a3fd1e] alter complete (version 1)
I20260926 07:35:25.059131 11754 raft_consensus.cc:348] T aa02661e7531402687f03e86dbc19418 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.062520 11754 raft_consensus.cc:374] T aa02661e7531402687f03e86dbc19418 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:25.062806 11754 raft_consensus.cc:729] T aa02661e7531402687f03e86dbc19418 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:25.023288 11401 leader_election.cc:302] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:25.019935 11842 raft_consensus.cc:504] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.062891 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 43) took 1095 ms. Trace:
I20260926 07:35:25.064538 11845 raft_consensus.cc:2781] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:25.065016 11845 raft_consensus.cc:686] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:25.064250 11752 rpcz_store.cc:276] 0926 07:35:23.967775 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:23.967954 (+ 179us) service_pool.cc:227] Handling call
0926 07:35:24.826707 (+858753us) ts_tablet_manager.cc:1145] Deleting tablet b7b89fcab5954a3aa1250be4a14a7bda
0926 07:35:24.826744 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:24.924317 (+ 97573us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:25.019886 (+ 95569us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:25.062830 (+ 42944us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":857786}
I20260926 07:35:25.066241 11845 consensus_queue.cc:237] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.066359 11754 consensus_queue.cc:260] T aa02661e7531402687f03e86dbc19418 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.067888 11823 raft_consensus.cc:482] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:25.068555 11823 raft_consensus.cc:504] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.072603 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 44) took 1079 ms. Trace:
W20260926 07:35:25.072872 11401 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.1:41829, user_credentials={real_user=slave}} blocked reactor thread for 49663.6us
I20260926 07:35:25.072908 11752 rpcz_store.cc:276] 0926 07:35:23.986552 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:23.986677 (+ 125us) service_pool.cc:227] Handling call
0926 07:35:25.065639 (+1078962us) ts_tablet_manager.cc:1145] Deleting tablet acc5568dee9b473f8f5f1f8bc3b5504e
0926 07:35:25.065675 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:25.065906 (+ 231us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1078438}
I20260926 07:35:25.073853 11746 tablet_bootstrap.cc:492] T aa02661e7531402687f03e86dbc19418 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
W20260926 07:35:25.075585 11546 leader_election.cc:341] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Not found: Tablet not found: 7000bcd55a984d64ba9d7d50fe63822c
I20260926 07:35:25.078541 11842 leader_election.cc:288] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:25.078671 11611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cc53f05536743b8a24ab2c88937f8f2" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:25.079234 11611 raft_consensus.cc:3037] T 8cc53f05536743b8a24ab2c88937f8f2 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:25.084077 11752 tablet_replica.cc:330] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:25.084506 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7000bcd55a984d64ba9d7d50fe63822c" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:25.085125 11752 raft_consensus.cc:2229] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:25.085070 11466 raft_consensus.cc:2454] T 7000bcd55a984d64ba9d7d50fe63822c P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 0.
I20260926 07:35:25.085781 11752 pending_rounds.cc:62] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1: Trying to abort 1 pending ops.
I20260926 07:35:25.086007 11752 pending_rounds.cc:68] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333511680283492352 op_type: NO_OP noop_request { }
I20260926 07:35:25.086139 11545 leader_election.cc:302] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters: e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:25.086369 11752 raft_consensus.cc:2866] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 07:35:25.086647 11752 raft_consensus.cc:2258] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:25.087874 11820 raft_consensus.cc:2781] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260926 07:35:25.087870 11401 leader_election.cc:341] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Not found: Tablet not found: 8cc53f05536743b8a24ab2c88937f8f2
I20260926 07:35:25.088189 11820 raft_consensus.cc:482] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:25.088466 11820 raft_consensus.cc:3037] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:25.089627 11746 tablet_bootstrap.cc:654] T aa02661e7531402687f03e86dbc19418 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:25.090261 11823 leader_election.cc:288] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:25.099294 11752 ts_tablet_manager.cc:1905] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 07:35:25.102068 11111 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4239f5d48512403bb711ece401f0756b has no consensus state
I20260926 07:35:25.070431 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet b7b89fcab5954a3aa1250be4a14a7bda (table default.Table_582802febc8147cba686fb42d81b56df [id=10eb71ca30de4a7da89de44126029c68]) successfully deleted
W20260926 07:35:25.107306 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet acc5568dee9b473f8f5f1f8bc3b5504e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet acc5568dee9b473f8f5f1f8bc3b5504e already in progress: creating tablet
I20260926 07:35:25.108644 11117 master_service.cc:946] ReplaceTablet: received request to replace tablet aa02661e7531402687f03e86dbc19418 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:25.107831 11754 ts_tablet_manager.cc:1428] T aa02661e7531402687f03e86dbc19418 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.051s user 0.004s sys 0.002s
I20260926 07:35:25.110147 11754 tablet_bootstrap.cc:492] T 345d14fc9eea46ad9b1155d538118ee5 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:25.114116 11117 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_65217d931d594a70b45c7d5c486930ba" } modify_external_catalogs: true
I20260926 07:35:25.115782 11754 tablet_bootstrap.cc:654] T 345d14fc9eea46ad9b1155d538118ee5 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:25.134840 11611 raft_consensus.cc:2454] T 8cc53f05536743b8a24ab2c88937f8f2 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:25.136655 11402 leader_election.cc:302] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters: e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:25.137812 11842 raft_consensus.cc:2781] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:25.138193 11842 raft_consensus.cc:686] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:25.139015 11842 consensus_queue.cc:237] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.157795 11117 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_65217d931d594a70b45c7d5c486930ba" } modify_external_catalogs: true
W20260926 07:35:25.117491 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 1e1cea4523a64428aac03ca0bdf708c7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1e1cea4523a64428aac03ca0bdf708c7 already in progress: creating tablet
I20260926 07:35:25.166255 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet afa901cdc6a84220bf36ee669a955b72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:24 UTC) from {username='slave'} at 127.0.0.1:40244
W20260926 07:35:25.121587 11120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_de31b81bd131450d913e95011f68aa2d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:35:25.116716 11113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4fc371a1e3de4170846c211c5d8ed72f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:35:25.189956 11119 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a0bdf6a4656e4217b0a7013fddc4cca8 consensus state has no leader
I20260926 07:35:25.232693 11753 ts_tablet_manager.cc:1918] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:25.233022 11753 log.cc:1196] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/50d0e4cd192c445bb7c82d1a3c4a53d1
I20260926 07:35:25.233608 11753 ts_tablet_manager.cc:1939] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:25.235651 11753 tablet_replica.cc:330] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:25.235845 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet 50d0e4cd192c445bb7c82d1a3c4a53d1 (table default.Table_b8d5ad40dd0b48609fb6831f16cd7fb6 [id=31cfe82ee0424be9bb920fa81e2a22f0]) successfully deleted
I20260926 07:35:25.236405 11820 raft_consensus.cc:504] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.238157 11820 leader_election.cc:288] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:25.242249 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7000bcd55a984d64ba9d7d50fe63822c" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:25.243037 11466 raft_consensus.cc:3037] T 7000bcd55a984d64ba9d7d50fe63822c P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:25.243067 11833 raft_consensus.cc:482] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:25.243458 11833 raft_consensus.cc:504] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
W20260926 07:35:25.244707 11546 leader_election.cc:341] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Not found: Tablet not found: 7000bcd55a984d64ba9d7d50fe63822c
I20260926 07:35:25.245185 11833 leader_election.cc:288] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:25.247545 11756 tablet_bootstrap.cc:492] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:25.247918 11756 ts_tablet_manager.cc:1397] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.391s user 0.046s sys 0.057s
I20260926 07:35:25.248593 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345d14fc9eea46ad9b1155d538118ee5" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:25.249182 11465 raft_consensus.cc:2454] T 345d14fc9eea46ad9b1155d538118ee5 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 0.
I20260926 07:35:25.250063 11756 raft_consensus.cc:348] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.250315 11247 leader_election.cc:302] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:25.250682 11756 raft_consensus.cc:374] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:25.250905 11756 raft_consensus.cc:729] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:25.251106 11833 raft_consensus.cc:2781] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:25.251392 11833 raft_consensus.cc:482] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:25.251709 11833 raft_consensus.cc:3037] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:25.251422 11756 consensus_queue.cc:260] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.252776 11746 tablet_bootstrap.cc:492] T aa02661e7531402687f03e86dbc19418 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:25.253579 11746 ts_tablet_manager.cc:1397] T aa02661e7531402687f03e86dbc19418 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.180s user 0.019s sys 0.040s
I20260926 07:35:25.253238 11611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345d14fc9eea46ad9b1155d538118ee5" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
W20260926 07:35:25.254776 11249 leader_election.cc:341] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:25.257844 11753 raft_consensus.cc:2229] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:25.258304 11753 raft_consensus.cc:2258] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:25.260311 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_4eb22a54d0df4f808fe85886a5809ab7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:25.266119 11753 ts_tablet_manager.cc:1905] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:25.255792 11746 raft_consensus.cc:348] T aa02661e7531402687f03e86dbc19418 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.272209 11746 raft_consensus.cc:374] T aa02661e7531402687f03e86dbc19418 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260926 07:35:25.272184 11113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4eb22a54d0df4f808fe85886a5809ab7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:25.272455 11746 raft_consensus.cc:729] T aa02661e7531402687f03e86dbc19418 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:25.270071 11756 ts_tablet_manager.cc:1428] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.022s user 0.009s sys 0.000s
I20260926 07:35:25.275751 11756 tablet_bootstrap.cc:492] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:25.274843 11746 consensus_queue.cc:260] T aa02661e7531402687f03e86dbc19418 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.278501 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet 7000bcd55a984d64ba9d7d50fe63822c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:25.281903 11756 tablet_bootstrap.cc:654] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:25.273159 11857 raft_consensus.cc:482] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:25.282974 11857 raft_consensus.cc:504] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.285815 11746 ts_tablet_manager.cc:1428] T aa02661e7531402687f03e86dbc19418 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.032s user 0.003s sys 0.003s
I20260926 07:35:25.288311 11754 tablet_bootstrap.cc:492] T 345d14fc9eea46ad9b1155d538118ee5 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:25.289840 11754 ts_tablet_manager.cc:1397] T 345d14fc9eea46ad9b1155d538118ee5 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.180s user 0.020s sys 0.061s
I20260926 07:35:25.293111 11857 leader_election.cc:288] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:25.293052 11611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa02661e7531402687f03e86dbc19418" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:25.293632 11611 raft_consensus.cc:2454] T aa02661e7531402687f03e86dbc19418 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 0.
I20260926 07:35:25.295614 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa02661e7531402687f03e86dbc19418" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:25.296288 11465 raft_consensus.cc:2454] T aa02661e7531402687f03e86dbc19418 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 0.
I20260926 07:35:25.298272 11247 leader_election.cc:302] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:25.298945 11857 raft_consensus.cc:2781] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:25.299242 11857 raft_consensus.cc:482] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:25.299520 11857 raft_consensus.cc:3037] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:25.299288 11746 tablet_bootstrap.cc:492] T afa901cdc6a84220bf36ee669a955b72 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:25.300403 11754 raft_consensus.cc:348] T 345d14fc9eea46ad9b1155d538118ee5 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.301100 11754 raft_consensus.cc:374] T 345d14fc9eea46ad9b1155d538118ee5 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:25.301383 11754 raft_consensus.cc:729] T 345d14fc9eea46ad9b1155d538118ee5 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:25.302052 11754 consensus_queue.cc:260] T 345d14fc9eea46ad9b1155d538118ee5 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.303987 11120 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_a84cdaff0c794776a40dc68d90b7ce79"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:25.307046 11119 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_fdc97586e1b74502987e6062bfed73d8" } new_table_name: "default.Table_f9178d5e3d934f1f9ef267a782316458" modify_external_catalogs: true
W20260926 07:35:25.311205 11120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a84cdaff0c794776a40dc68d90b7ce79 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:25.314069 11746 tablet_bootstrap.cc:654] T afa901cdc6a84220bf36ee669a955b72 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:25.315478 11754 ts_tablet_manager.cc:1428] T 345d14fc9eea46ad9b1155d538118ee5 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.025s user 0.004s sys 0.000s
I20260926 07:35:25.318795 11756 tablet_bootstrap.cc:492] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:25.319162 11756 ts_tablet_manager.cc:1397] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.044s user 0.008s sys 0.001s
I20260926 07:35:25.321208 11756 raft_consensus.cc:348] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.321818 11756 raft_consensus.cc:374] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:25.322055 11756 raft_consensus.cc:729] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:25.322294 11116 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_de0d2fdff98146f9aede70ab6a51dbec" } modify_external_catalogs: true
I20260926 07:35:25.322809 11116 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_de0d2fdff98146f9aede70ab6a51dbec" } modify_external_catalogs: true
I20260926 07:35:25.325572 11746 tablet_bootstrap.cc:492] T afa901cdc6a84220bf36ee669a955b72 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:25.325932 11746 ts_tablet_manager.cc:1397] T afa901cdc6a84220bf36ee669a955b72 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.028s user 0.008s sys 0.000s
I20260926 07:35:25.327704 11754 tablet_bootstrap.cc:492] T f31a069dc888487ebbbb1cb91e886ad0 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:25.327965 11746 raft_consensus.cc:348] T afa901cdc6a84220bf36ee669a955b72 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.328604 11746 raft_consensus.cc:374] T afa901cdc6a84220bf36ee669a955b72 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:25.328830 11746 raft_consensus.cc:729] T afa901cdc6a84220bf36ee669a955b72 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:25.322767 11756 consensus_queue.cc:260] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.330740 11752 ts_tablet_manager.cc:1918] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:25.331084 11752 log.cc:1196] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/50d0e4cd192c445bb7c82d1a3c4a53d1
I20260926 07:35:25.330399 11756 ts_tablet_manager.cc:1428] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260926 07:35:25.331712 11752 ts_tablet_manager.cc:1939] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:25.331967 11756 tablet_bootstrap.cc:492] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:25.335341 11466 raft_consensus.cc:2454] T 7000bcd55a984d64ba9d7d50fe63822c P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 1.
I20260926 07:35:25.336654 11545 leader_election.cc:302] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters: e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:25.336558 11833 raft_consensus.cc:504] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.337661 11820 raft_consensus.cc:2781] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:25.338058 11820 raft_consensus.cc:686] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Becoming Leader. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Running, Role: LEADER
I20260926 07:35:25.338346 11833 leader_election.cc:288] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:25.338776 11820 consensus_queue.cc:237] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
W20260926 07:35:25.348655 11119 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ae16af9edff9401589bcf54eeb3e8dfe has no consensus state
I20260926 07:35:25.350286 11611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345d14fc9eea46ad9b1155d538118ee5" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:25.350811 11756 tablet_bootstrap.cc:654] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:25.350795 11611 raft_consensus.cc:3037] T 345d14fc9eea46ad9b1155d538118ee5 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:25.355466 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345d14fc9eea46ad9b1155d538118ee5" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:25.357635 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet 345d14fc9eea46ad9b1155d538118ee5 with delete_type TABLET_DATA_DELETED (replaced by tablet 97d02d87a4d34974b9f1d5409511f62c) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:25.359543 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cc53f05536743b8a24ab2c88937f8f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 9a255ef54f7e410ca8a779e5d7a118ca) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:25.359138 11746 consensus_queue.cc:260] T afa901cdc6a84220bf36ee669a955b72 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.355965 11466 raft_consensus.cc:3037] T 345d14fc9eea46ad9b1155d538118ee5 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:25.369889 11756 tablet_bootstrap.cc:492] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:25.370239 11756 ts_tablet_manager.cc:1397] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.038s user 0.003s sys 0.006s
I20260926 07:35:25.370456 11752 tablet_replica.cc:330] T 7000bcd55a984d64ba9d7d50fe63822c P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:25.371131 11752 raft_consensus.cc:2229] T 7000bcd55a984d64ba9d7d50fe63822c P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:25.371524 11752 raft_consensus.cc:2258] T 7000bcd55a984d64ba9d7d50fe63822c P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:25.372239 11756 raft_consensus.cc:348] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.372707 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet 50d0e4cd192c445bb7c82d1a3c4a53d1 (table default.Table_b8d5ad40dd0b48609fb6831f16cd7fb6 [id=31cfe82ee0424be9bb920fa81e2a22f0]) successfully deleted
I20260926 07:35:25.372851 11756 raft_consensus.cc:374] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:25.373078 11756 raft_consensus.cc:729] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
W20260926 07:35:25.373322 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet e06ed286144941269b9a0c0c60bff0ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e06ed286144941269b9a0c0c60bff0ce already in progress: creating tablet
W20260926 07:35:25.373917 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet afa901cdc6a84220bf36ee669a955b72 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet afa901cdc6a84220bf36ee669a955b72 already in progress: creating tablet
I20260926 07:35:25.375895 11752 ts_tablet_manager.cc:1905] T 7000bcd55a984d64ba9d7d50fe63822c P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:25.383296 11746 ts_tablet_manager.cc:1428] T afa901cdc6a84220bf36ee669a955b72 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.057s user 0.006s sys 0.000s
I20260926 07:35:25.386840 11756 consensus_queue.cc:260] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.388741 11756 ts_tablet_manager.cc:1428] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.018s user 0.004s sys 0.001s
I20260926 07:35:25.391492 11746 tablet_bootstrap.cc:492] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:25.393911 11114 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_4fc371a1e3de4170846c211c5d8ed72f" } new_table_name: "default.Table_53678f9c4c01403eb2d846d3242be0aa" modify_external_catalogs: true
I20260926 07:35:25.389494 11756 tablet_bootstrap.cc:492] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:25.403599 11746 tablet_bootstrap.cc:654] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:25.413614 11754 tablet_bootstrap.cc:654] T f31a069dc888487ebbbb1cb91e886ad0 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:25.418869 11756 tablet_bootstrap.cc:654] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:25.422909 11118 catalog_manager.cc:5495] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade reported cstate change: term changed from 0 to 1, leader changed from <none> to 6070e9e3d1234a9d83e49431416afade (127.10.171.3). New cstate: current_term: 1 leader_uuid: "6070e9e3d1234a9d83e49431416afade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } health_report { overall_health: UNKNOWN } } }
I20260926 07:35:25.424361 11118 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet a000cb9928de4459be1e6b1b7a81848a (table default.Table_d0380e9d3f26439bbc91cf476aeae69f [id=19875953f2b6484f934503404af57593]). Expected version 2 got 0
I20260926 07:35:25.424883 11754 tablet_bootstrap.cc:492] T f31a069dc888487ebbbb1cb91e886ad0 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:25.425323 11754 ts_tablet_manager.cc:1397] T f31a069dc888487ebbbb1cb91e886ad0 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.100s user 0.011s sys 0.004s
I20260926 07:35:25.426908 11115 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet f31a069dc888487ebbbb1cb91e886ad0 (table default.Table_eb3d1f18577c4064b04bceab813f25e6 [id=47d4200172794cc78a6f386fee4b15b4]). Expected version 1 got 0
I20260926 07:35:25.428864 11754 raft_consensus.cc:348] T f31a069dc888487ebbbb1cb91e886ad0 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.429786 11754 raft_consensus.cc:374] T f31a069dc888487ebbbb1cb91e886ad0 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:25.432858 11292 tablet_service.cc:1514] Processing DeleteTablet for tablet 345d14fc9eea46ad9b1155d538118ee5 with delete_type TABLET_DATA_DELETED (replaced by tablet 97d02d87a4d34974b9f1d5409511f62c) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:25.437215 11756 tablet_bootstrap.cc:492] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:25.437572 11756 ts_tablet_manager.cc:1397] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.048s user 0.005s sys 0.004s
I20260926 07:35:25.438174 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_3231dafd497f4c8e8a13703613affde3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:25.438621 11857 raft_consensus.cc:504] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.439565 11756 raft_consensus.cc:348] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
W20260926 07:35:25.439971 11113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3231dafd497f4c8e8a13703613affde3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:25.440322 11756 raft_consensus.cc:374] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:25.440562 11756 raft_consensus.cc:729] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:25.441557 11746 tablet_bootstrap.cc:492] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:25.441956 11746 ts_tablet_manager.cc:1397] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.051s user 0.007s sys 0.001s
I20260926 07:35:25.444368 11466 raft_consensus.cc:2454] T 345d14fc9eea46ad9b1155d538118ee5 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 1.
I20260926 07:35:25.445621 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet aa02661e7531402687f03e86dbc19418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:23 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:25.446450 11247 leader_election.cc:302] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:25.446806 11756 consensus_queue.cc:260] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.448279 11833 raft_consensus.cc:482] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:25.447999 11115 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_dca4d96797ae4d008e202dea5b1c364d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:25.448773 11833 raft_consensus.cc:504] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
W20260926 07:35:25.450464 11115 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dca4d96797ae4d008e202dea5b1c364d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:25.452975 11746 raft_consensus.cc:348] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.454023 11746 raft_consensus.cc:374] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:25.454269 11746 raft_consensus.cc:729] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:25.454612 11833 leader_election.cc:288] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:25.452013 11858 raft_consensus.cc:2781] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:25.454963 11746 consensus_queue.cc:260] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.455458 11858 raft_consensus.cc:686] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Becoming Leader. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Running, Role: LEADER
I20260926 07:35:25.456167 11858 consensus_queue.cc:237] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.460358 11111 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet a0bdf6a4656e4217b0a7013fddc4cca8 (table default.Table_c7fb27daeba04d90bc0cf0c7ac07fe56 [id=3ce63f9fb5ad4b82ba2fbbf2e3678271]). Expected version 1 got 0
I20260926 07:35:25.461117 11292 tablet_service.cc:1514] Processing DeleteTablet for tablet aa02661e7531402687f03e86dbc19418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:23 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:25.464799 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa901cdc6a84220bf36ee669a955b72" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:25.465619 11466 raft_consensus.cc:2454] T afa901cdc6a84220bf36ee669a955b72 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 0.
I20260926 07:35:25.468321 11857 leader_election.cc:288] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:25.471438 11247 leader_election.cc:302] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:25.476495 11746 ts_tablet_manager.cc:1428] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.034s user 0.007s sys 0.001s
I20260926 07:35:25.476195 11442 tablet_service.cc:1467] Processing CreateTablet for tablet 640d21586bad4070957cff61bd174fd2 (DEFAULT_TABLE table=default.Table_c961864a78e341a9bf757eaebadc4e84 [id=fa5001d12cb14306b0a5ab0f0c79bc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:25.480145 11292 tablet_service.cc:1467] Processing CreateTablet for tablet 640d21586bad4070957cff61bd174fd2 (DEFAULT_TABLE table=default.Table_c961864a78e341a9bf757eaebadc4e84 [id=fa5001d12cb14306b0a5ab0f0c79bc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:25.482753 11443 tablet_service.cc:1467] Processing CreateTablet for tablet 55ad1d96e2314d1eb0a717a3e9ff31bc (DEFAULT_TABLE table=default.Table_4d463023afd047528c16b7ee4dabd36f [id=eeb221378f35425eb8d24b815f8a2181]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:25.484598 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa02661e7531402687f03e86dbc19418" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:25.485281 11466 raft_consensus.cc:3037] T aa02661e7531402687f03e86dbc19418 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:25.487982 11746 tablet_bootstrap.cc:492] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:25.493234 11746 tablet_bootstrap.cc:654] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:25.507233 11857 raft_consensus.cc:2781] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:25.507683 11857 raft_consensus.cc:482] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:25.508088 11857 raft_consensus.cc:3037] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:25.513617 11833 raft_consensus.cc:482] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:25.514020 11833 raft_consensus.cc:504] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.518278 11833 leader_election.cc:288] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:25.520056 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f31a069dc888487ebbbb1cb91e886ad0" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
W20260926 07:35:25.522639 11247 leader_election.cc:341] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:25.533560 11752 ts_tablet_manager.cc:1918] T 7000bcd55a984d64ba9d7d50fe63822c P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:25.533912 11752 log.cc:1196] T 7000bcd55a984d64ba9d7d50fe63822c P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/7000bcd55a984d64ba9d7d50fe63822c
I20260926 07:35:25.534531 11752 ts_tablet_manager.cc:1939] T 7000bcd55a984d64ba9d7d50fe63822c P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:25.535225 11442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 640d21586bad4070957cff61bd174fd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:25.536543 11443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55ad1d96e2314d1eb0a717a3e9ff31bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:25.542877 11288 tablet_service.cc:1467] Processing CreateTablet for tablet 55ad1d96e2314d1eb0a717a3e9ff31bc (DEFAULT_TABLE table=default.Table_4d463023afd047528c16b7ee4dabd36f [id=eeb221378f35425eb8d24b815f8a2181]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:25.545409 11753 ts_tablet_manager.cc:1918] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:25.545785 11752 tablet_replica.cc:330] T 345d14fc9eea46ad9b1155d538118ee5 P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:25.545742 11753 log.cc:1196] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/cb27378f5ed24e76804b05d5fb1cf441
I20260926 07:35:25.546420 11752 raft_consensus.cc:2229] T 345d14fc9eea46ad9b1155d538118ee5 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:25.546811 11752 raft_consensus.cc:2258] T 345d14fc9eea46ad9b1155d538118ee5 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:25.547305 11753 ts_tablet_manager.cc:1939] T cb27378f5ed24e76804b05d5fb1cf441 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:25.445192 11611 raft_consensus.cc:2454] T 345d14fc9eea46ad9b1155d538118ee5 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 1.
I20260926 07:35:25.453953 11610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa901cdc6a84220bf36ee669a955b72" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:25.470547 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa02661e7531402687f03e86dbc19418" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:25.549497 11609 raft_consensus.cc:3037] T aa02661e7531402687f03e86dbc19418 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:25.549896 11752 ts_tablet_manager.cc:1905] T 345d14fc9eea46ad9b1155d538118ee5 P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:25.473453 11583 tablet_service.cc:1467] Processing CreateTablet for tablet 640d21586bad4070957cff61bd174fd2 (DEFAULT_TABLE table=default.Table_c961864a78e341a9bf757eaebadc4e84 [id=fa5001d12cb14306b0a5ab0f0c79bc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:25.552793 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 640d21586bad4070957cff61bd174fd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:25.480327 11585 tablet_service.cc:1467] Processing CreateTablet for tablet 55ad1d96e2314d1eb0a717a3e9ff31bc (DEFAULT_TABLE table=default.Table_4d463023afd047528c16b7ee4dabd36f [id=eeb221378f35425eb8d24b815f8a2181]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:25.512292 11820 consensus_queue.cc:1035] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:25.557089 11585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55ad1d96e2314d1eb0a717a3e9ff31bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:25.516940 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f31a069dc888487ebbbb1cb91e886ad0" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:25.589792 11754 raft_consensus.cc:729] T f31a069dc888487ebbbb1cb91e886ad0 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:25.590569 11754 consensus_queue.cc:260] T f31a069dc888487ebbbb1cb91e886ad0 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.591506 11608 raft_consensus.cc:2454] T f31a069dc888487ebbbb1cb91e886ad0 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 0.
I20260926 07:35:25.596697 11586 tablet_service.cc:1467] Processing CreateTablet for tablet d6fc6178f1e54c3984a20873864233cd (DEFAULT_TABLE table=default.Table_b98681120d654cb4bfd9fb67e73e3b74 [id=f299d82b1a01434690a378d2494fd370]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:25.597873 11586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fc6178f1e54c3984a20873864233cd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:25.507802 11114 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 034d508f0a164f3ea8308ea2d28818c4 has no consensus state
I20260926 07:35:25.548341 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet 7000bcd55a984d64ba9d7d50fe63822c (table default.Table_4bc696441dce4bc687a0344b93eccb9e [id=12a935394c1f49158f105705ce038532]) successfully deleted
I20260926 07:35:25.552894 11119 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_de31b81bd131450d913e95011f68aa2d" } new_table_name: "default.Table_0f44ba35f0544fdfa9ad17c5be72ecad" modify_external_catalogs: true
I20260926 07:35:25.600493 11114 master_service.cc:946] ReplaceTablet: received request to replace tablet afa901cdc6a84220bf36ee669a955b72 from {username='slave'} at 127.0.0.1:44836
W20260926 07:35:25.600773 11105 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.171.2:42379, user_credentials={real_user=slave}} blocked reactor thread for 52435us
I20260926 07:35:25.601785 11584 tablet_service.cc:1467] Processing CreateTablet for tablet ae16af9edff9401589bcf54eeb3e8dfe (DEFAULT_TABLE table=default.Table_f9178d5e3d934f1f9ef267a782316458 [id=0c18461963f444269787b667b3648709]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:25.603183 11584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae16af9edff9401589bcf54eeb3e8dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:25.603899 11442 tablet_service.cc:1467] Processing CreateTablet for tablet d6fc6178f1e54c3984a20873864233cd (DEFAULT_TABLE table=default.Table_b98681120d654cb4bfd9fb67e73e3b74 [id=f299d82b1a01434690a378d2494fd370]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:25.604674 11441 tablet_service.cc:1467] Processing CreateTablet for tablet ae16af9edff9401589bcf54eeb3e8dfe (DEFAULT_TABLE table=default.Table_f9178d5e3d934f1f9ef267a782316458 [id=0c18461963f444269787b667b3648709]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:25.606112 11445 tablet_service.cc:1514] Processing DeleteTablet for tablet afa901cdc6a84220bf36ee669a955b72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:24 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:25.607096 11445 tablet_service.cc:1514] Processing DeleteTablet for tablet acc5568dee9b473f8f5f1f8bc3b5504e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:23 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:25.620282 11445 tablet_service.cc:1467] Processing CreateTablet for tablet a8b629a4aba045b697e33bbebecb05b7 (DEFAULT_TABLE table=default.Table_de0d2fdff98146f9aede70ab6a51dbec [id=c469a15ec981495a879b63ec5f961277]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:25.620781 11590 tablet_service.cc:1467] Processing CreateTablet for tablet a8b629a4aba045b697e33bbebecb05b7 (DEFAULT_TABLE table=default.Table_de0d2fdff98146f9aede70ab6a51dbec [id=c469a15ec981495a879b63ec5f961277]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:25.622021 11590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8b629a4aba045b697e33bbebecb05b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:25.625946 11746 tablet_bootstrap.cc:492] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:25.626331 11746 ts_tablet_manager.cc:1397] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.140s user 0.007s sys 0.005s
I20260926 07:35:25.628402 11746 raft_consensus.cc:348] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.629127 11117 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 509f607b089c44bc9c3f80ed59a68c95
I20260926 07:35:25.629205 11746 raft_consensus.cc:374] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:25.629477 11746 raft_consensus.cc:729] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:25.631592 11746 consensus_queue.cc:260] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.632007 11112 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 067ae4ea43f54e88874d93b4393dae27
I20260926 07:35:25.633394 11112 catalog_manager.cc:6565] ReplaceTablet: tablet 067ae4ea43f54e88874d93b4393dae27 deleted and replaced by tablet 0218d2a727f942e48d770878e6bf4a54
I20260926 07:35:25.636762 11587 tablet_service.cc:1514] Processing DeleteTablet for tablet 509f607b089c44bc9c3f80ed59a68c95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:25.547657 11292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 640d21586bad4070957cff61bd174fd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:25.641711 11439 tablet_service.cc:1514] Processing DeleteTablet for tablet 067ae4ea43f54e88874d93b4393dae27 with delete_type TABLET_DATA_DELETED (replaced by tablet 0218d2a727f942e48d770878e6bf4a54) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:25.644536 11120 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_607b0d428b034d258b62fabe5e45de97"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:25.645052 11591 tablet_service.cc:1514] Processing DeleteTablet for tablet 067ae4ea43f54e88874d93b4393dae27 with delete_type TABLET_DATA_DELETED (replaced by tablet 0218d2a727f942e48d770878e6bf4a54) from {username='slave'} at 127.0.0.1:35244
W20260926 07:35:25.645949 11120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_607b0d428b034d258b62fabe5e45de97 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:25.648847 11442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fc6178f1e54c3984a20873864233cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:25.648618 11820 consensus_queue.cc:1035] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 07:35:25.650446 11441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae16af9edff9401589bcf54eeb3e8dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:25.652911 11443 tablet_service.cc:1514] Processing DeleteTablet for tablet 509f607b089c44bc9c3f80ed59a68c95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:25.656503 11445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8b629a4aba045b697e33bbebecb05b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:25.659839 11746 ts_tablet_manager.cc:1428] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.033s user 0.007s sys 0.000s
I20260926 07:35:25.660696 11746 tablet_bootstrap.cc:492] T 067ae4ea43f54e88874d93b4393dae27 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:25.662374 11288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55ad1d96e2314d1eb0a717a3e9ff31bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:25.686893 11746 tablet_bootstrap.cc:654] T 067ae4ea43f54e88874d93b4393dae27 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:25.687886 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 509f607b089c44bc9c3f80ed59a68c95 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 509f607b089c44bc9c3f80ed59a68c95 already in progress: creating tablet
W20260926 07:35:25.688254 11104 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:37629: connect: Connection refused (error 111) [suppressed 11 similar messages]
W20260926 07:35:25.689364 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 067ae4ea43f54e88874d93b4393dae27 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 067ae4ea43f54e88874d93b4393dae27 already in progress: creating tablet
I20260926 07:35:25.696143 11443 tablet_service.cc:1467] Processing CreateTablet for tablet b7837bc0821a4b68972b663606364955 (DEFAULT_TABLE table=default.Table_582802febc8147cba686fb42d81b56df [id=10eb71ca30de4a7da89de44126029c68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:25.697238 11443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7837bc0821a4b68972b663606364955. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:25.569100 11249 leader_election.cc:341] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:25.617483 11287 tablet_service.cc:1467] Processing CreateTablet for tablet d6fc6178f1e54c3984a20873864233cd (DEFAULT_TABLE table=default.Table_b98681120d654cb4bfd9fb67e73e3b74 [id=f299d82b1a01434690a378d2494fd370]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:25.707530 11287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fc6178f1e54c3984a20873864233cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:25.619470 11293 tablet_service.cc:1467] Processing CreateTablet for tablet ae16af9edff9401589bcf54eeb3e8dfe (DEFAULT_TABLE table=default.Table_f9178d5e3d934f1f9ef267a782316458 [id=0c18461963f444269787b667b3648709]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:25.712813 11293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae16af9edff9401589bcf54eeb3e8dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:25.633541 11295 tablet_service.cc:1514] Processing DeleteTablet for tablet 14e5da3711fe46c4bb46a7a60f41a581 with delete_type TABLET_DATA_DELETED (replaced by tablet 35f8ea999baa4d38a1433975ff6ccbb5) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:25.634908 11294 tablet_service.cc:1514] Processing DeleteTablet for tablet 509f607b089c44bc9c3f80ed59a68c95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:25.635874 11289 tablet_service.cc:1514] Processing DeleteTablet for tablet 067ae4ea43f54e88874d93b4393dae27 with delete_type TABLET_DATA_DELETED (replaced by tablet 0218d2a727f942e48d770878e6bf4a54) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:25.718326 11587 tablet_service.cc:1467] Processing CreateTablet for tablet b7837bc0821a4b68972b663606364955 (DEFAULT_TABLE table=default.Table_582802febc8147cba686fb42d81b56df [id=10eb71ca30de4a7da89de44126029c68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:25.719406 11587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7837bc0821a4b68972b663606364955. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:25.726898 11249 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.3:45127, user_credentials={real_user=slave}} blocked reactor thread for 157849us
I20260926 07:35:25.727661 11249 leader_election.cc:302] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1
I20260926 07:35:25.734412 11746 tablet_bootstrap.cc:492] T 067ae4ea43f54e88874d93b4393dae27 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:25.734831 11746 ts_tablet_manager.cc:1397] T 067ae4ea43f54e88874d93b4393dae27 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.074s user 0.008s sys 0.000s
I20260926 07:35:25.737787 11609 raft_consensus.cc:2454] T aa02661e7531402687f03e86dbc19418 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 1.
I20260926 07:35:25.746901 11746 raft_consensus.cc:348] T 067ae4ea43f54e88874d93b4393dae27 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.750105 11746 raft_consensus.cc:374] T 067ae4ea43f54e88874d93b4393dae27 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:25.750380 11746 raft_consensus.cc:729] T 067ae4ea43f54e88874d93b4393dae27 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:25.751482 11858 raft_consensus.cc:2781] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:25.751799 11858 raft_consensus.cc:482] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:25.752100 11858 raft_consensus.cc:3037] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:25.757153 11249 leader_election.cc:302] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:25.761548 11833 raft_consensus.cc:482] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:25.761945 11833 raft_consensus.cc:504] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.764171 11833 leader_election.cc:288] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:25.764760 11833 raft_consensus.cc:2781] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:25.765163 11833 raft_consensus.cc:686] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Becoming Leader. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Running, Role: LEADER
I20260926 07:35:25.767184 11464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "067ae4ea43f54e88874d93b4393dae27" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:25.775010 11752 ts_tablet_manager.cc:1918] T 345d14fc9eea46ad9b1155d538118ee5 P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:25.775013 11288 tablet_service.cc:1467] Processing CreateTablet for tablet a8b629a4aba045b697e33bbebecb05b7 (DEFAULT_TABLE table=default.Table_de0d2fdff98146f9aede70ab6a51dbec [id=c469a15ec981495a879b63ec5f961277]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:25.775373 11752 log.cc:1196] T 345d14fc9eea46ad9b1155d538118ee5 P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/345d14fc9eea46ad9b1155d538118ee5
I20260926 07:35:25.776057 11752 ts_tablet_manager.cc:1939] T 345d14fc9eea46ad9b1155d538118ee5 P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:25.776044 11288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8b629a4aba045b697e33bbebecb05b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:25.776301 11746 consensus_queue.cc:260] T 067ae4ea43f54e88874d93b4393dae27 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.777245 11464 raft_consensus.cc:2454] T 067ae4ea43f54e88874d93b4393dae27 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 0.
I20260926 07:35:25.782872 11247 leader_election.cc:302] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:25.791360 11754 ts_tablet_manager.cc:1428] T f31a069dc888487ebbbb1cb91e886ad0 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.366s user 0.004s sys 0.004s
I20260926 07:35:25.792249 11754 tablet_bootstrap.cc:492] T 067ae4ea43f54e88874d93b4393dae27 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:25.794492 11746 ts_tablet_manager.cc:1428] T 067ae4ea43f54e88874d93b4393dae27 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.059s user 0.004s sys 0.004s
I20260926 07:35:25.795321 11746 tablet_bootstrap.cc:492] T 14e5da3711fe46c4bb46a7a60f41a581 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:25.807773 11289 tablet_service.cc:1467] Processing CreateTablet for tablet b7837bc0821a4b68972b663606364955 (DEFAULT_TABLE table=default.Table_582802febc8147cba686fb42d81b56df [id=10eb71ca30de4a7da89de44126029c68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:25.809098 11289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7837bc0821a4b68972b663606364955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:25.810298 11746 tablet_bootstrap.cc:654] T 14e5da3711fe46c4bb46a7a60f41a581 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:25.821403 11114 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_b98681120d654cb4bfd9fb67e73e3b74" } new_table_name: "default.Table_051aa34eea804c698486395b4bf2553f" modify_external_catalogs: true
I20260926 07:35:25.794780 11833 consensus_queue.cc:237] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.825479 11111 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_a97a2f44056d4080b8eb45cb95106a2f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:25.827152 11111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a97a2f44056d4080b8eb45cb95106a2f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:25.835541 11112 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 1620b507f07c446c8c1811895a097409 (table default.Table_0a660d82b80c40d8a4c41b9abdec566f [id=f23a7ed0439c41149de764ab244ef60b]). Expected version 2 got 0
I20260926 07:35:25.841936 11112 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_4d463023afd047528c16b7ee4dabd36f" } modify_external_catalogs: true
I20260926 07:35:25.842456 11112 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_4d463023afd047528c16b7ee4dabd36f" } modify_external_catalogs: true
I20260926 07:35:25.849135 11858 raft_consensus.cc:504] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:25.853350 11753 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 53) took 1426 ms. Trace:
I20260926 07:35:25.853837 11753 rpcz_store.cc:276] 0926 07:35:24.426962 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:24.427052 (+ 90us) service_pool.cc:227] Handling call
0926 07:35:25.235524 (+808472us) ts_tablet_manager.cc:1145] Deleting tablet cb27378f5ed24e76804b05d5fb1cf441
0926 07:35:25.235560 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:25.440404 (+204844us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:25.545210 (+104806us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:25.853247 (+308037us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":807966}
W20260926 07:35:25.855569 11104 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 55ad1d96e2314d1eb0a717a3e9ff31bc (table default.Table_4d463023afd047528c16b7ee4dabd36f [id=eeb221378f35425eb8d24b815f8a2181], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 55ad1d96e2314d1eb0a717a3e9ff31bc consensus state has no leader
I20260926 07:35:25.855818 11753 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 56) took 1049 ms. Trace:
I20260926 07:35:25.857182 11464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f31a069dc888487ebbbb1cb91e886ad0" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
W20260926 07:35:25.858748 11247 leader_election.cc:341] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:25.869335 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet cb27378f5ed24e76804b05d5fb1cf441 (table default.Table_cb7f69c004f64db7bdc4412924ca513e [id=4c1477cba48a45558d6e0dfe08cd8afa]) successfully deleted
I20260926 07:35:25.870857 11878 raft_consensus.cc:2781] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:25.871200 11878 raft_consensus.cc:482] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:25.871507 11878 raft_consensus.cc:3037] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
W20260926 07:35:25.872874 11249 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.3:45127, user_credentials={real_user=slave}} blocked reactor thread for 115884us
I20260926 07:35:25.874300 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "067ae4ea43f54e88874d93b4393dae27" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:25.851130 11858 leader_election.cc:288] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
W20260926 07:35:25.884793 11118 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.3:53335 (request call id 23) took 1181 ms (client timeout 1000 ms). Trace:
I20260926 07:35:25.885164 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f31a069dc888487ebbbb1cb91e886ad0" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:25.885758 11609 raft_consensus.cc:3037] T f31a069dc888487ebbbb1cb91e886ad0 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:25.890771 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet 345d14fc9eea46ad9b1155d538118ee5 (table default.Table_c1c4fe0ecbe840e59d8ba4f659432e0d [id=4c9f2ba8e0db427da572f73f02d4fa9e]) successfully deleted
W20260926 07:35:25.885213 11118 rpcz_store.cc:269] 0926 07:35:24.702948 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:24.712013 (+ 9065us) service_pool.cc:227] Handling call
0926 07:35:25.884713 (+1172700us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:25.654833 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:25.655197 (+ 364us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:25.655222 (+ 25us) write_op.cc:435] Acquired schema lock
0926 07:35:25.655235 (+ 13us) tablet.cc:660] Decoding operations
0926 07:35:25.655490 (+ 255us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:25.655526 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:25.655547 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0926 07:35:25.655683 (+ 136us) tablet.cc:699] Row locks acquired
0926 07:35:25.655697 (+ 14us) write_op.cc:260] PREPARE: finished
0926 07:35:25.655792 (+ 95us) write_op.cc:270] Start()
0926 07:35:25.655883 (+ 91us) write_op.cc:276] Timestamp: P: 1790408125655752 usec, L: 0
0926 07:35:25.655900 (+ 17us) op_driver.cc:331] REPLICATION: starting
0926 07:35:25.822536 (+166636us) log.cc:842] Serialized 529 byte log entry
0926 07:35:25.823634 (+ 1098us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032b471 00007fb8103a2198 00007fb810803ade 00007fb8108032b8 00007fb810807ba1 00007fb810807b59 00007fb810807ae9 00007fb810807ab1 00007fb810806dad 0000000000580784 00000000005805b9
0926 07:35:25.879369 (+ 55735us) op_driver.cc:444] REPLICATION: finished
0926 07:35:25.880691 (+ 1322us) write_op.cc:301] APPLY: starting
0926 07:35:25.880743 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:25.880823 (+ 80us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:25.880842 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:25.880970 (+ 128us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:25.881102 (+ 132us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:25.881140 (+ 38us) write_op.cc:312] APPLY: finished
0926 07:35:25.881278 (+ 138us) log.cc:842] Serialized 22 byte log entry
0926 07:35:25.881388 (+ 110us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:25.881440 (+ 52us) write_op.cc:454] Released schema lock
0926 07:35:25.881468 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:25.881651 (+ 183us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:25.881674 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":1190,"apply.run_cpu_time_us":1120,"apply.run_wall_time_us":1123,"num_ops":1,"prepare.queue_time_us":228647,"prepare.run_cpu_time_us":4400,"prepare.run_wall_time_us":169484,"raft.queue_time_us":11890,"raft.run_cpu_time_us":790,"raft.run_wall_time_us":789,"replication_time_us":223398,"spinlock_wait_cycles":381802880}]]}
I20260926 07:35:25.904351 11116 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet a8b629a4aba045b697e33bbebecb05b7
I20260926 07:35:25.907860 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet a8b629a4aba045b697e33bbebecb05b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:25.908255 11587 tablet_service.cc:1514] Processing DeleteTablet for tablet a8b629a4aba045b697e33bbebecb05b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:25.908588 11752 tablet_replica.cc:330] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:25.909276 11752 raft_consensus.cc:2229] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:25.909897 11752 pending_rounds.cc:62] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1: Trying to abort 1 pending ops.
W20260926 07:35:25.909970 11107 catalog_manager.cc:4808] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet a8b629a4aba045b697e33bbebecb05b7 because the tablet was not found. No further retry: Not found: Tablet not found: a8b629a4aba045b697e33bbebecb05b7
I20260926 07:35:25.910094 11752 pending_rounds.cc:68] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333511680588156928 op_type: NO_OP noop_request { }
I20260926 07:35:25.910452 11752 raft_consensus.cc:2866] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 07:35:25.910766 11752 raft_consensus.cc:2258] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:25.911433 11756 ts_tablet_manager.cc:1428] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.474s user 0.012s sys 0.076s
I20260926 07:35:25.912356 11756 tablet_bootstrap.cc:492] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:25.912927 11752 ts_tablet_manager.cc:1905] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 07:35:25.931203 11119 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3e28ca159d8349748812825ac1947c51 has no consensus state
I20260926 07:35:25.931766 11113 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet a000cb9928de4459be1e6b1b7a81848a (table default.Table_d0380e9d3f26439bbc91cf476aeae69f [id=19875953f2b6484f934503404af57593]). Expected version 2 got 0
I20260926 07:35:25.941906 11118 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet a000cb9928de4459be1e6b1b7a81848a (table default.Table_d0380e9d3f26439bbc91cf476aeae69f [id=19875953f2b6484f934503404af57593]). Expected version 2 got 0
I20260926 07:35:25.943076 11118 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet f31a069dc888487ebbbb1cb91e886ad0 (table default.Table_eb3d1f18577c4064b04bceab813f25e6 [id=47d4200172794cc78a6f386fee4b15b4]). Expected version 1 got 0
I20260926 07:35:25.947172 11756 tablet_bootstrap.cc:654] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:25.953629 11507 diagnostics_log.cc:230] Unable to collect stacks to diagnostics log: Incomplete: not collecting stack trace since debugger or strace is attached
I20260926 07:35:25.960428 11466 raft_consensus.cc:2454] T aa02661e7531402687f03e86dbc19418 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 1.
W20260926 07:35:25.892540 11249 leader_election.cc:341] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:25.986673 11754 tablet_bootstrap.cc:654] T 067ae4ea43f54e88874d93b4393dae27 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:25.989300 11746 tablet_bootstrap.cc:492] T 14e5da3711fe46c4bb46a7a60f41a581 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
W20260926 07:35:25.963032 11114 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d6fc6178f1e54c3984a20873864233cd consensus state has no leader
I20260926 07:35:25.973735 11753 rpcz_store.cc:276] 0926 07:35:24.806539 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:24.926285 (+119746us) service_pool.cc:227] Handling call
0926 07:35:25.855440 (+929155us) ts_tablet_manager.cc:1145] Deleting tablet e06ed286144941269b9a0c0c60bff0ce
0926 07:35:25.855486 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:25.855764 (+ 278us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":928540}
I20260926 07:35:25.967568 11112 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 55ad1d96e2314d1eb0a717a3e9ff31bc
I20260926 07:35:25.995639 11753 tablet_replica.cc:330] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:25.999162 11294 tablet_service.cc:1514] Processing DeleteTablet for tablet 55ad1d96e2314d1eb0a717a3e9ff31bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:26.001720 11288 tablet_service.cc:1514] Processing DeleteTablet for tablet a8b629a4aba045b697e33bbebecb05b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:52070
W20260926 07:35:26.001684 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet e06ed286144941269b9a0c0c60bff0ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e06ed286144941269b9a0c0c60bff0ce already in progress: creating tablet
I20260926 07:35:25.996093 11587 tablet_service.cc:1514] Processing DeleteTablet for tablet 55ad1d96e2314d1eb0a717a3e9ff31bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:35244
W20260926 07:35:26.006378 11107 catalog_manager.cc:4808] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 55ad1d96e2314d1eb0a717a3e9ff31bc because the tablet was not found. No further retry: Not found: Tablet not found: 55ad1d96e2314d1eb0a717a3e9ff31bc
I20260926 07:35:26.020507 11756 tablet_bootstrap.cc:492] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
W20260926 07:35:26.020413 11249 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.3:45127, user_credentials={real_user=slave}} blocked reactor thread for 127912us
I20260926 07:35:26.020931 11756 ts_tablet_manager.cc:1397] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.109s user 0.008s sys 0.004s
I20260926 07:35:26.023463 11756 raft_consensus.cc:348] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:26.033051 11756 raft_consensus.cc:374] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:26.033366 11756 raft_consensus.cc:729] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:26.039983 11756 consensus_queue.cc:260] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:26.044045 11756 ts_tablet_manager.cc:1428] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.023s user 0.002s sys 0.003s
I20260926 07:35:26.054072 11587 tablet_service.cc:1514] Processing DeleteTablet for tablet 345d14fc9eea46ad9b1155d538118ee5 with delete_type TABLET_DATA_DELETED (replaced by tablet 97d02d87a4d34974b9f1d5409511f62c) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:26.054891 11871 tablet_replica.cc:330] T 345d14fc9eea46ad9b1155d538118ee5 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:26.055667 11871 raft_consensus.cc:2229] T 345d14fc9eea46ad9b1155d538118ee5 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:26.056134 11871 raft_consensus.cc:2258] T 345d14fc9eea46ad9b1155d538118ee5 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:26.057996 11871 ts_tablet_manager.cc:1905] T 345d14fc9eea46ad9b1155d538118ee5 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:26.058663 11609 raft_consensus.cc:2454] T f31a069dc888487ebbbb1cb91e886ad0 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 1.
W20260926 07:35:26.059537 11857 consensus_meta.cc:287] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent flushing consensus metadata: real 0.551s user 0.007s sys 0.001s
I20260926 07:35:26.059883 11857 raft_consensus.cc:504] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:26.061897 11857 leader_election.cc:288] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:26.072183 11753 raft_consensus.cc:2229] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:26.073809 11753 raft_consensus.cc:2258] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:26.075845 11753 ts_tablet_manager.cc:1905] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:26.079471 11249 leader_election.cc:302] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1
I20260926 07:35:26.080458 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa901cdc6a84220bf36ee669a955b72" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:26.081303 11857 raft_consensus.cc:2781] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:26.081413 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cc53f05536743b8a24ab2c88937f8f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 9a255ef54f7e410ca8a779e5d7a118ca) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:26.081879 11857 raft_consensus.cc:686] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Becoming Leader. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Running, Role: LEADER
I20260926 07:35:26.082067 11590 tablet_service.cc:1514] Processing DeleteTablet for tablet 7000bcd55a984d64ba9d7d50fe63822c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:26.080098 11584 tablet_service.cc:1514] Processing DeleteTablet for tablet 50d0e4cd192c445bb7c82d1a3c4a53d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:22 UTC) from {username='slave'} at 127.0.0.1:35244
W20260926 07:35:26.083227 11585 consensus_meta.cc:287] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade: Time spent flushing consensus metadata: real 0.500s user 0.010s sys 0.000s
I20260926 07:35:26.082803 11857 consensus_queue.cc:237] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
W20260926 07:35:26.085243 11249 leader_election.cc:341] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:26.086151 11586 tablet_service.cc:1514] Processing DeleteTablet for tablet aa02661e7531402687f03e86dbc19418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:23 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:25.989861 11746 ts_tablet_manager.cc:1397] T 14e5da3711fe46c4bb46a7a60f41a581 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.195s user 0.008s sys 0.003s
I20260926 07:35:26.044900 11756 tablet_bootstrap.cc:492] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:26.104828 11746 raft_consensus.cc:348] T 14e5da3711fe46c4bb46a7a60f41a581 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:26.106079 11746 raft_consensus.cc:374] T 14e5da3711fe46c4bb46a7a60f41a581 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:26.106388 11746 raft_consensus.cc:729] T 14e5da3711fe46c4bb46a7a60f41a581 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:26.107128 11857 raft_consensus.cc:482] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:26.107097 11746 consensus_queue.cc:260] T 14e5da3711fe46c4bb46a7a60f41a581 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:26.107568 11857 raft_consensus.cc:504] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
W20260926 07:35:26.110282 11546 consensus_peers.cc:487] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7000bcd55a984d64ba9d7d50fe63822c. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:26.110785 11441 tablet_service.cc:1514] Processing DeleteTablet for tablet 55ad1d96e2314d1eb0a717a3e9ff31bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:26.111694 11112 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_607b0d428b034d258b62fabe5e45de97" } modify_external_catalogs: true
I20260926 07:35:26.113183 11112 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_607b0d428b034d258b62fabe5e45de97" } modify_external_catalogs: true
I20260926 07:35:26.116739 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0bdf6a4656e4217b0a7013fddc4cca8" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:26.117031 11746 ts_tablet_manager.cc:1428] T 14e5da3711fe46c4bb46a7a60f41a581 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.016s user 0.001s sys 0.010s
W20260926 07:35:26.119385 11249 leader_election.cc:341] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:26.122864 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0bdf6a4656e4217b0a7013fddc4cca8" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
W20260926 07:35:26.124408 11247 leader_election.cc:341] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:26.124910 11247 leader_election.cc:302] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade
I20260926 07:35:26.125991 11833 raft_consensus.cc:2726] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:26.126116 11754 tablet_bootstrap.cc:492] T 067ae4ea43f54e88874d93b4393dae27 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:26.126521 11754 ts_tablet_manager.cc:1397] T 067ae4ea43f54e88874d93b4393dae27 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.334s user 0.008s sys 0.008s
I20260926 07:35:26.128742 11857 leader_election.cc:288] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:26.132531 11114 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_e08de921f78841ccb011f4082de0fbe0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:26.123360 11118 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_4c7ed90b32024fb888d7aef0e188ea80"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:26.135821 11111 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_0f44ba35f0544fdfa9ad17c5be72ecad" } new_table_name: "default.Table_9444fa8f3e7e4c7e9a245b34db0fb9f1" modify_external_catalogs: true
I20260926 07:35:26.138197 11112 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet d064cd317eb3488d95f4c6536176878d
I20260926 07:35:26.141952 11887 tablet.cc:1722] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260926 07:35:26.147710 11114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e08de921f78841ccb011f4082de0fbe0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:26.148322 11746 tablet_bootstrap.cc:492] T 9a255ef54f7e410ca8a779e5d7a118ca P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
W20260926 07:35:26.151750 11111 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3e28ca159d8349748812825ac1947c51 has no consensus state
W20260926 07:35:26.154585 11118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4c7ed90b32024fb888d7aef0e188ea80 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:26.155851 11842 raft_consensus.cc:482] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:26.156359 11842 raft_consensus.cc:504] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:26.159606 11842 leader_election.cc:288] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:26.159324 11111 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_c111fc58a37e47878582962cca421abe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:26.160972 11111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c111fc58a37e47878582962cca421abe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:26.169209 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_8304288ba9e54e32a49193d760c6e068"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:26.170943 11112 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8304288ba9e54e32a49193d760c6e068 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:26.163369 11857 raft_consensus.cc:482] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:26.174707 11756 tablet_bootstrap.cc:654] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:26.173605 11857 raft_consensus.cc:504] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:26.176299 11857 leader_election.cc:288] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:26.178434 11464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e5da3711fe46c4bb46a7a60f41a581" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:26.100863 11889 raft_consensus.cc:482] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:26.100932 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa901cdc6a84220bf36ee669a955b72" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:26.179360 11464 raft_consensus.cc:2454] T 14e5da3711fe46c4bb46a7a60f41a581 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 0.
I20260926 07:35:26.179802 11466 raft_consensus.cc:3037] T afa901cdc6a84220bf36ee669a955b72 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:26.184082 11247 leader_election.cc:302] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:26.185143 11857 raft_consensus.cc:2781] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:26.185474 11857 raft_consensus.cc:482] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:26.185775 11857 raft_consensus.cc:3037] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:26.179279 11889 raft_consensus.cc:504] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:26.187426 11889 leader_election.cc:288] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:26.198325 11114 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_dca4d96797ae4d008e202dea5b1c364d" } new_table_name: "default.Table_54b98a70017749499ab6c312bd7e6255" modify_external_catalogs: true
I20260926 07:35:26.203991 11120 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_051aa34eea804c698486395b4bf2553f" } modify_external_catalogs: true
I20260926 07:35:26.204710 11120 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_051aa34eea804c698486395b4bf2553f" } modify_external_catalogs: true
I20260926 07:35:26.207726 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acc5568dee9b473f8f5f1f8bc3b5504e" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:26.208987 11610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e5da3711fe46c4bb46a7a60f41a581" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
W20260926 07:35:26.211856 11249 leader_election.cc:341] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:26.166538 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1620b507f07c446c8c1811895a097409" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:26.220502 11754 raft_consensus.cc:348] T 067ae4ea43f54e88874d93b4393dae27 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:26.221354 11754 raft_consensus.cc:374] T 067ae4ea43f54e88874d93b4393dae27 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:26.222115 11754 raft_consensus.cc:729] T 067ae4ea43f54e88874d93b4393dae27 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
W20260926 07:35:26.228603 11402 leader_election.cc:341] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: must be running to vote when last-logged opid is not known
W20260926 07:35:26.229640 11402 leader_election.cc:341] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:26.230624 11746 tablet_bootstrap.cc:654] T 9a255ef54f7e410ca8a779e5d7a118ca P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:26.235453 11401 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.171.1:41829, user_credentials={real_user=slave}} blocked reactor thread for 103148us
I20260926 07:35:26.236403 11895 tablet.cc:1722] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 07:35:26.237512 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1620b507f07c446c8c1811895a097409" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:26.238586 11316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acc5568dee9b473f8f5f1f8bc3b5504e" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:26.239140 11316 raft_consensus.cc:2454] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
W20260926 07:35:26.240787 11401 leader_election.cc:341] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
W20260926 07:35:26.240825 11247 consensus_peers.cc:487] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e -> Peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Couldn't send request to peer 22f3902d01584936b20437606cb86dd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 345d14fc9eea46ad9b1155d538118ee5. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:26.241274 11401 leader_election.cc:302] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 22f3902d01584936b20437606cb86dd1; no voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:26.242193 11842 raft_consensus.cc:2726] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:26.243270 11401 leader_election.cc:302] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 6070e9e3d1234a9d83e49431416afade
I20260926 07:35:26.244093 11842 raft_consensus.cc:2781] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:26.244434 11842 raft_consensus.cc:482] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:26.244748 11842 raft_consensus.cc:3037] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:26.231518 11754 consensus_queue.cc:260] T 067ae4ea43f54e88874d93b4393dae27 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:26.247653 11886 tablet.cc:1722] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 07:35:26.261943 11878 raft_consensus.cc:504] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:26.264639 11871 ts_tablet_manager.cc:1918] T 345d14fc9eea46ad9b1155d538118ee5 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:26.264993 11871 log.cc:1196] T 345d14fc9eea46ad9b1155d538118ee5 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/345d14fc9eea46ad9b1155d538118ee5
I20260926 07:35:26.265362 11466 raft_consensus.cc:2454] T afa901cdc6a84220bf36ee669a955b72 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 1.
I20260926 07:35:26.265643 11871 ts_tablet_manager.cc:1939] T 345d14fc9eea46ad9b1155d538118ee5 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:26.274178 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "067ae4ea43f54e88874d93b4393dae27" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:26.274736 11609 raft_consensus.cc:3037] T 067ae4ea43f54e88874d93b4393dae27 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:26.279999 11754 ts_tablet_manager.cc:1428] T 067ae4ea43f54e88874d93b4393dae27 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.145s user 0.006s sys 0.001s
I20260926 07:35:26.285374 11746 tablet_bootstrap.cc:492] T 9a255ef54f7e410ca8a779e5d7a118ca P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:26.285771 11746 ts_tablet_manager.cc:1397] T 9a255ef54f7e410ca8a779e5d7a118ca P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.138s user 0.011s sys 0.001s
I20260926 07:35:26.289105 11247 leader_election.cc:302] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 6070e9e3d1234a9d83e49431416afade
I20260926 07:35:26.289155 11746 raft_consensus.cc:348] T 9a255ef54f7e410ca8a779e5d7a118ca P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:26.289786 11746 raft_consensus.cc:374] T 9a255ef54f7e410ca8a779e5d7a118ca P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260926 07:35:26.289796 11247 raft_consensus.cc:2670] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260926 07:35:26.290021 11746 raft_consensus.cc:729] T 9a255ef54f7e410ca8a779e5d7a118ca P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:26.292352 11754 tablet_bootstrap.cc:492] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:26.294627 11746 consensus_queue.cc:260] T 9a255ef54f7e410ca8a779e5d7a118ca P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:26.263988 11878 leader_election.cc:288] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
W20260926 07:35:26.298918 11114 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5cd482fb159d418291cb02bc9138b076 has no consensus state
I20260926 07:35:26.301963 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "067ae4ea43f54e88874d93b4393dae27" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:26.302714 11466 raft_consensus.cc:3037] T 067ae4ea43f54e88874d93b4393dae27 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:26.304549 11119 master_service.cc:946] ReplaceTablet: received request to replace tablet f31a069dc888487ebbbb1cb91e886ad0 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:26.307596 11111 master_service.cc:946] ReplaceTablet: received request to replace tablet 067ae4ea43f54e88874d93b4393dae27 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:26.312356 11756 tablet_bootstrap.cc:492] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:26.312783 11756 ts_tablet_manager.cc:1397] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.268s user 0.024s sys 0.004s
I20260926 07:35:26.315038 11756 raft_consensus.cc:348] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:26.315737 11756 raft_consensus.cc:374] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:26.315987 11756 raft_consensus.cc:729] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:26.317381 11754 tablet_bootstrap.cc:654] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:26.321285 11746 ts_tablet_manager.cc:1428] T 9a255ef54f7e410ca8a779e5d7a118ca P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.035s user 0.004s sys 0.001s
I20260926 07:35:26.320384 11756 consensus_queue.cc:260] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:26.329144 11756 ts_tablet_manager.cc:1428] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20260926 07:35:26.333027 11292 tablet_service.cc:1467] Processing CreateTablet for tablet c81a59361a6e42b7962b3e027c4a087f (DEFAULT_TABLE table=default.Table_a84cdaff0c794776a40dc68d90b7ce79 [id=707c7ee5ced14cee8b8005cce80dbbb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.332998 11441 tablet_service.cc:1467] Processing CreateTablet for tablet c81a59361a6e42b7962b3e027c4a087f (DEFAULT_TABLE table=default.Table_a84cdaff0c794776a40dc68d90b7ce79 [id=707c7ee5ced14cee8b8005cce80dbbb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.334996 11756 tablet_bootstrap.cc:492] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:26.335304 11586 tablet_service.cc:1467] Processing CreateTablet for tablet c81a59361a6e42b7962b3e027c4a087f (DEFAULT_TABLE table=default.Table_a84cdaff0c794776a40dc68d90b7ce79 [id=707c7ee5ced14cee8b8005cce80dbbb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.335397 11442 tablet_service.cc:1467] Processing CreateTablet for tablet 034d508f0a164f3ea8308ea2d28818c4 (DEFAULT_TABLE table=default.Table_53678f9c4c01403eb2d846d3242be0aa [id=db19d49c9c29488e89f7280e91a11713]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.335772 11293 tablet_service.cc:1467] Processing CreateTablet for tablet 034d508f0a164f3ea8308ea2d28818c4 (DEFAULT_TABLE table=default.Table_53678f9c4c01403eb2d846d3242be0aa [id=db19d49c9c29488e89f7280e91a11713]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.340958 11585 tablet_service.cc:1467] Processing CreateTablet for tablet 034d508f0a164f3ea8308ea2d28818c4 (DEFAULT_TABLE table=default.Table_53678f9c4c01403eb2d846d3242be0aa [id=db19d49c9c29488e89f7280e91a11713]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.341521 11289 tablet_service.cc:1467] Processing CreateTablet for tablet 0eae53ce278749208492815b918582f3 (DEFAULT_TABLE table=default.Table_03b976b95c404da1b2b14a6d71dd8a72 [id=f66bffa952604b28bb51299de795ee2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.341593 11445 tablet_service.cc:1467] Processing CreateTablet for tablet 0eae53ce278749208492815b918582f3 (DEFAULT_TABLE table=default.Table_03b976b95c404da1b2b14a6d71dd8a72 [id=f66bffa952604b28bb51299de795ee2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.343777 11118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_0a660d82b80c40d8a4c41b9abdec566f" } modify_external_catalogs: true
I20260926 07:35:26.344424 11118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_0a660d82b80c40d8a4c41b9abdec566f" } modify_external_catalogs: true
I20260926 07:35:26.347421 11584 tablet_service.cc:1467] Processing CreateTablet for tablet 0eae53ce278749208492815b918582f3 (DEFAULT_TABLE table=default.Table_03b976b95c404da1b2b14a6d71dd8a72 [id=f66bffa952604b28bb51299de795ee2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.349702 11754 tablet_bootstrap.cc:492] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:26.349516 11287 tablet_service.cc:1467] Processing CreateTablet for tablet c36c759963cb4915afdca7d12109cc6e (DEFAULT_TABLE table=default.Table_b9f92e070aab45cdb8352c1d8a3d603c [id=7245a652e96a445fa9ae35517ac03adf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.349568 11443 tablet_service.cc:1467] Processing CreateTablet for tablet c36c759963cb4915afdca7d12109cc6e (DEFAULT_TABLE table=default.Table_b9f92e070aab45cdb8352c1d8a3d603c [id=7245a652e96a445fa9ae35517ac03adf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.350075 11754 ts_tablet_manager.cc:1397] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.058s user 0.008s sys 0.000s
I20260926 07:35:26.354840 11754 raft_consensus.cc:348] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:26.355548 11754 raft_consensus.cc:374] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:26.355811 11754 raft_consensus.cc:729] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:26.357007 11756 tablet_bootstrap.cc:654] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:26.360083 11584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eae53ce278749208492815b918582f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.361917 11112 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_eb3d1f18577c4064b04bceab813f25e6" } new_table_name: "default.Table_ba08930510174e65840452201a15f719" modify_external_catalogs: true
I20260926 07:35:26.362733 11753 ts_tablet_manager.cc:1918] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:26.363040 11753 log.cc:1196] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/afa901cdc6a84220bf36ee669a955b72
I20260926 07:35:26.364282 11746 tablet_bootstrap.cc:492] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:26.364634 11752 ts_tablet_manager.cc:1918] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:26.364997 11752 log.cc:1196] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/8cc53f05536743b8a24ab2c88937f8f2
I20260926 07:35:26.365839 11752 ts_tablet_manager.cc:1939] T 8cc53f05536743b8a24ab2c88937f8f2 P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:26.366331 11609 raft_consensus.cc:2454] T 067ae4ea43f54e88874d93b4393dae27 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 1.
I20260926 07:35:26.370819 11842 raft_consensus.cc:504] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:26.373430 11120 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet d6fc6178f1e54c3984a20873864233cd
I20260926 07:35:26.376276 11439 tablet_service.cc:1514] Processing DeleteTablet for tablet d6fc6178f1e54c3984a20873864233cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:26 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:26.379716 11441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c81a59361a6e42b7962b3e027c4a087f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.357987 11754 consensus_queue.cc:260] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:26.382498 11590 tablet_service.cc:1514] Processing DeleteTablet for tablet d6fc6178f1e54c3984a20873864233cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:26 UTC) from {username='slave'} at 127.0.0.1:35244
W20260926 07:35:26.383545 11545 consensus_peers.cc:487] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade -> Peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Couldn't send request to peer 22f3902d01584936b20437606cb86dd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7000bcd55a984d64ba9d7d50fe63822c. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:26.385699 11442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 034d508f0a164f3ea8308ea2d28818c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.387001 11443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c36c759963cb4915afdca7d12109cc6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.388733 11445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eae53ce278749208492815b918582f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.389360 11119 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet f31a069dc888487ebbbb1cb91e886ad0
I20260926 07:35:26.366348 11586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c81a59361a6e42b7962b3e027c4a087f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.390475 11585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 034d508f0a164f3ea8308ea2d28818c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.390586 11119 catalog_manager.cc:6565] ReplaceTablet: tablet f31a069dc888487ebbbb1cb91e886ad0 deleted and replaced by tablet 248ab84a1f284c2bbddf62f00b485e13
I20260926 07:35:26.391103 11753 ts_tablet_manager.cc:1939] T afa901cdc6a84220bf36ee669a955b72 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:26.391597 11871 tablet_replica.cc:330] T 8cc53f05536743b8a24ab2c88937f8f2 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:26.391808 11113 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet a000cb9928de4459be1e6b1b7a81848a (table default.Table_d0380e9d3f26439bbc91cf476aeae69f [id=19875953f2b6484f934503404af57593]). Expected version 2 got 0
I20260926 07:35:26.392341 11871 raft_consensus.cc:2229] T 8cc53f05536743b8a24ab2c88937f8f2 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:26.392769 11871 raft_consensus.cc:2258] T 8cc53f05536743b8a24ab2c88937f8f2 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:26.393736 11117 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 1620b507f07c446c8c1811895a097409 (table default.Table_0a660d82b80c40d8a4c41b9abdec566f [id=f23a7ed0439c41149de764ab244ef60b]). Expected version 2 got 0
I20260926 07:35:26.394249 11117 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet a000cb9928de4459be1e6b1b7a81848a (table default.Table_d0380e9d3f26439bbc91cf476aeae69f [id=19875953f2b6484f934503404af57593]). Expected version 2 got 0
I20260926 07:35:26.397795 11842 leader_election.cc:288] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:26.400409 11754 ts_tablet_manager.cc:1428] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.050s user 0.007s sys 0.000s
I20260926 07:35:26.401211 11754 tablet_bootstrap.cc:492] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:26.405356 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 345d14fc9eea46ad9b1155d538118ee5 (table default.Table_c1c4fe0ecbe840e59d8ba4f659432e0d [id=4c9f2ba8e0db427da572f73f02d4fa9e]) successfully deleted
I20260926 07:35:26.406595 11439 tablet_service.cc:1514] Processing DeleteTablet for tablet f31a069dc888487ebbbb1cb91e886ad0 with delete_type TABLET_DATA_DELETED (replaced by tablet 248ab84a1f284c2bbddf62f00b485e13) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:26.407776 11440 tablet_service.cc:1467] Processing CreateTablet for tablet 7e52ec7d802a4e90bccef777f77a2a1b (DEFAULT_TABLE table=default.Table_ba9d6249f2594f7687befc65703d5f25 [id=f7c3dc1977564d65a4b44fb8bf7eeabb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.408485 11857 raft_consensus.cc:504] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:26.409029 11444 tablet_service.cc:1467] Processing CreateTablet for tablet 3e28ca159d8349748812825ac1947c51 (DEFAULT_TABLE table=default.Table_9444fa8f3e7e4c7e9a245b34db0fb9f1 [id=cc4663f9d1734d9cb4cc4e1f061310a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.410280 11857 leader_election.cc:288] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:26.410849 11249 leader_election.cc:302] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:26.411580 11292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c81a59361a6e42b7962b3e027c4a087f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.414032 11439 tablet_service.cc:1467] Processing CreateTablet for tablet 04be706330924282961054c5835cec73 (DEFAULT_TABLE table=default.Table_4eb22a54d0df4f808fe85886a5809ab7 [id=cab439115f3c46d089824983fe26d0eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.415079 11446 tablet_service.cc:1467] Processing CreateTablet for tablet 04180ace5d4548dfbcd226bf229d5c26 (DEFAULT_TABLE table=default.Table_2f42e2e9196745bb942de35a1373e040 [id=abb24fc02fd44eb98163e5c7ad16cf63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.416404 11111 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_74db930591fe400780cd8ffcfb3ae89d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:26.417994 11111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_74db930591fe400780cd8ffcfb3ae89d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:26.420469 11288 tablet_service.cc:1514] Processing DeleteTablet for tablet d6fc6178f1e54c3984a20873864233cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:26 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:26.420387 11290 tablet_service.cc:1467] Processing CreateTablet for tablet 04180ace5d4548dfbcd226bf229d5c26 (DEFAULT_TABLE table=default.Table_2f42e2e9196745bb942de35a1373e040 [id=abb24fc02fd44eb98163e5c7ad16cf63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.420549 11291 tablet_service.cc:1467] Processing CreateTablet for tablet 04be706330924282961054c5835cec73 (DEFAULT_TABLE table=default.Table_4eb22a54d0df4f808fe85886a5809ab7 [id=cab439115f3c46d089824983fe26d0eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.421471 11316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acc5568dee9b473f8f5f1f8bc3b5504e" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:26.422003 11316 raft_consensus.cc:3037] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:26.424379 11294 tablet_service.cc:1514] Processing DeleteTablet for tablet f31a069dc888487ebbbb1cb91e886ad0 with delete_type TABLET_DATA_DELETED (replaced by tablet 248ab84a1f284c2bbddf62f00b485e13) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:26.425777 11295 tablet_service.cc:1467] Processing CreateTablet for tablet 7e52ec7d802a4e90bccef777f77a2a1b (DEFAULT_TABLE table=default.Table_ba9d6249f2594f7687befc65703d5f25 [id=f7c3dc1977564d65a4b44fb8bf7eeabb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.427296 11296 tablet_service.cc:1467] Processing CreateTablet for tablet 3e28ca159d8349748812825ac1947c51 (DEFAULT_TABLE table=default.Table_9444fa8f3e7e4c7e9a245b34db0fb9f1 [id=cc4663f9d1734d9cb4cc4e1f061310a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.428089 11287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c36c759963cb4915afdca7d12109cc6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.429709 11293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 034d508f0a164f3ea8308ea2d28818c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.431735 11289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eae53ce278749208492815b918582f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.433630 11438 tablet_service.cc:1467] Processing CreateTablet for tablet 0218d2a727f942e48d770878e6bf4a54 (DEFAULT_TABLE table=default.Table_bc4fc15cb6014a1e86ba2886b2950bae [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.436190 11437 tablet_service.cc:1467] Processing CreateTablet for tablet 4239f5d48512403bb711ece401f0756b (DEFAULT_TABLE table=default.Table_686ba040299440d0af09a9fe3501a040 [id=a639ca032a8d4fcdb928635eaa6db0e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.438862 11290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04180ace5d4548dfbcd226bf229d5c26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.439700 11288 tablet_service.cc:1467] Processing CreateTablet for tablet 4239f5d48512403bb711ece401f0756b (DEFAULT_TABLE table=default.Table_686ba040299440d0af09a9fe3501a040 [id=a639ca032a8d4fcdb928635eaa6db0e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.440855 11294 tablet_service.cc:1467] Processing CreateTablet for tablet 0218d2a727f942e48d770878e6bf4a54 (DEFAULT_TABLE table=default.Table_bc4fc15cb6014a1e86ba2886b2950bae [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.441644 11436 tablet_service.cc:1467] Processing CreateTablet for tablet 90f62593b6f844fc8a41b9be240101fd (DEFAULT_TABLE table=default.Table_904e49ad16464cdbac83d37d3cdf115b [id=b62d16bd5d214bb5bae010b4eae39c19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.443876 11857 raft_consensus.cc:2781] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:26.445636 11857 raft_consensus.cc:686] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Becoming Leader. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Running, Role: LEADER
I20260926 07:35:26.446194 11435 tablet_service.cc:1467] Processing CreateTablet for tablet 9feaf1ed8ff34928b716fd0b4e41cdde (DEFAULT_TABLE table=default.Table_783e2f568d774b5eb908860af5fa4166 [id=d5913e299402470898908b32cdba262a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.448438 11857 consensus_queue.cc:237] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:26.451429 11464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e5da3711fe46c4bb46a7a60f41a581" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:26.452147 11464 raft_consensus.cc:3037] T 14e5da3711fe46c4bb46a7a60f41a581 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:26.469099 11440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e52ec7d802a4e90bccef777f77a2a1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.469731 11436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90f62593b6f844fc8a41b9be240101fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.474474 11444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e28ca159d8349748812825ac1947c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.476408 11446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04180ace5d4548dfbcd226bf229d5c26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.477303 11746 tablet_bootstrap.cc:654] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:26.477730 11437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4239f5d48512403bb711ece401f0756b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.479146 11438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0218d2a727f942e48d770878e6bf4a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.480434 11439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04be706330924282961054c5835cec73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.488457 11435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9feaf1ed8ff34928b716fd0b4e41cdde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.499941 11746 tablet_bootstrap.cc:492] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:26.509559 11746 ts_tablet_manager.cc:1397] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.154s user 0.010s sys 0.001s
I20260926 07:35:26.516067 11746 raft_consensus.cc:348] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:26.544243 11746 raft_consensus.cc:374] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:26.544664 11746 raft_consensus.cc:729] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:26.546941 11746 consensus_queue.cc:260] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:26.551216 11291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04be706330924282961054c5835cec73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.554363 11746 ts_tablet_manager.cc:1428] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.041s user 0.002s sys 0.005s
I20260926 07:35:26.562047 11753 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 57) took 1623 ms. Trace:
I20260926 07:35:26.555928 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 55) took 1196 ms. Trace:
I20260926 07:35:26.562410 11753 rpcz_store.cc:276] 0926 07:35:24.938025 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:24.952309 (+ 14284us) service_pool.cc:227] Handling call
0926 07:35:25.995467 (+1043158us) ts_tablet_manager.cc:1145] Deleting tablet afa901cdc6a84220bf36ee669a955b72
0926 07:35:25.995515 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:26.073611 (+ 78096us) spinlock_profiling.cc:243] Waited 75.7 ms on lock 0x7b5400110154. stack: 00007fedc0a39033 00007fedc0a38f30 00007fedc05ba534 00000000004db2b4 00000000004db249 00007fedc7945471 00007fedc79c8352 00007fedc7dde26c 00007fedc89ebcfa 00007fedc8a1be10 00007fedc8a02e0d 00007fedc8a02db9 00007fedc8a02d49 00007fedc8a02d11 00007fedc8a01f3d 0000000000580784
0926 07:35:26.161100 (+ 87489us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:26.362493 (+201393us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:26.561428 (+198935us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":174059776,"tablet-delete.queue_time_us":1035736}
I20260926 07:35:26.562718 11752 rpcz_store.cc:276] 0926 07:35:25.359285 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:25.359406 (+ 121us) service_pool.cc:227] Handling call
0926 07:35:25.908417 (+549011us) ts_tablet_manager.cc:1145] Deleting tablet 8cc53f05536743b8a24ab2c88937f8f2
0926 07:35:25.908468 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:26.263738 (+355270us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:26.364342 (+100604us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:26.555860 (+191518us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":533376}
I20260926 07:35:26.563835 11466 raft_consensus.cc:2454] T 067ae4ea43f54e88874d93b4393dae27 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 1.
I20260926 07:35:26.564464 11288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4239f5d48512403bb711ece401f0756b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.564950 11752 tablet_replica.cc:330] T aa02661e7531402687f03e86dbc19418 P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:26.565828 11752 raft_consensus.cc:2229] T aa02661e7531402687f03e86dbc19418 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:26.566064 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet 8cc53f05536743b8a24ab2c88937f8f2 (table default.Table_836a28372ec4439d98b1583a9b50d2c1 [id=800a96cbd0b34639bbe7ad317a640053]) successfully deleted
I20260926 07:35:26.566339 11752 raft_consensus.cc:2258] T aa02661e7531402687f03e86dbc19418 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:26.568331 11294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0218d2a727f942e48d770878e6bf4a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.569931 11583 tablet_service.cc:1467] Processing CreateTablet for tablet c36c759963cb4915afdca7d12109cc6e (DEFAULT_TABLE table=default.Table_b9f92e070aab45cdb8352c1d8a3d603c [id=7245a652e96a445fa9ae35517ac03adf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.571201 11587 tablet_service.cc:1467] Processing CreateTablet for tablet 7e52ec7d802a4e90bccef777f77a2a1b (DEFAULT_TABLE table=default.Table_ba9d6249f2594f7687befc65703d5f25 [id=f7c3dc1977564d65a4b44fb8bf7eeabb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.576519 11754 tablet_bootstrap.cc:654] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:26.581389 11591 tablet_service.cc:1467] Processing CreateTablet for tablet 04180ace5d4548dfbcd226bf229d5c26 (DEFAULT_TABLE table=default.Table_2f42e2e9196745bb942de35a1373e040 [id=abb24fc02fd44eb98163e5c7ad16cf63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.586926 11752 ts_tablet_manager.cc:1905] T aa02661e7531402687f03e86dbc19418 P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:26.589201 11590 tablet_service.cc:1514] Processing DeleteTablet for tablet f31a069dc888487ebbbb1cb91e886ad0 with delete_type TABLET_DATA_DELETED (replaced by tablet 248ab84a1f284c2bbddf62f00b485e13) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:26.594760 11746 tablet_bootstrap.cc:492] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:26.612293 11582 tablet_service.cc:1467] Processing CreateTablet for tablet 4239f5d48512403bb711ece401f0756b (DEFAULT_TABLE table=default.Table_686ba040299440d0af09a9fe3501a040 [id=a639ca032a8d4fcdb928635eaa6db0e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.627502 11746 tablet_bootstrap.cc:654] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:26.629055 11589 tablet_service.cc:1467] Processing CreateTablet for tablet 3e28ca159d8349748812825ac1947c51 (DEFAULT_TABLE table=default.Table_9444fa8f3e7e4c7e9a245b34db0fb9f1 [id=cc4663f9d1734d9cb4cc4e1f061310a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.630239 11588 tablet_service.cc:1467] Processing CreateTablet for tablet 0218d2a727f942e48d770878e6bf4a54 (DEFAULT_TABLE table=default.Table_bc4fc15cb6014a1e86ba2886b2950bae [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.635718 11610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e5da3711fe46c4bb46a7a60f41a581" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:26.639676 11590 tablet_service.cc:1467] Processing CreateTablet for tablet 04be706330924282961054c5835cec73 (DEFAULT_TABLE table=default.Table_4eb22a54d0df4f808fe85886a5809ab7 [id=cab439115f3c46d089824983fe26d0eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.640739 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c36c759963cb4915afdca7d12109cc6e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:26.640846 11249 leader_election.cc:341] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:26.643105 11296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e28ca159d8349748812825ac1947c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.646070 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acc5568dee9b473f8f5f1f8bc3b5504e" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
W20260926 07:35:26.647640 11402 leader_election.cc:341] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:26.649405 11589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e28ca159d8349748812825ac1947c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.651993 11588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0218d2a727f942e48d770878e6bf4a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.657624 11286 tablet_service.cc:1467] Processing CreateTablet for tablet 90f62593b6f844fc8a41b9be240101fd (DEFAULT_TABLE table=default.Table_904e49ad16464cdbac83d37d3cdf115b [id=b62d16bd5d214bb5bae010b4eae39c19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.659564 11285 tablet_service.cc:1467] Processing CreateTablet for tablet 9feaf1ed8ff34928b716fd0b4e41cdde (DEFAULT_TABLE table=default.Table_783e2f568d774b5eb908860af5fa4166 [id=d5913e299402470898908b32cdba262a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.662990 11295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e52ec7d802a4e90bccef777f77a2a1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.663743 11286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90f62593b6f844fc8a41b9be240101fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.665230 11753 tablet_replica.cc:330] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:26.665918 11753 raft_consensus.cc:2229] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:26.666694 11753 pending_rounds.cc:62] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e: Trying to abort 1 pending ops.
I20260926 07:35:26.666895 11753 pending_rounds.cc:68] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333511681876045824 op_type: NO_OP noop_request { }
I20260926 07:35:26.667307 11753 raft_consensus.cc:2866] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 07:35:26.667608 11753 raft_consensus.cc:2258] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:26.667624 11285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9feaf1ed8ff34928b716fd0b4e41cdde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.675484 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet afa901cdc6a84220bf36ee669a955b72 (table default.Table_b5e830fcbaa44c209d50a84038d3b68f [id=be6bd29e04f342cbb7993b18e13d9332]) successfully deleted
I20260926 07:35:26.670064 11753 ts_tablet_manager.cc:1905] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:26.676821 11582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4239f5d48512403bb711ece401f0756b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.682727 11590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04be706330924282961054c5835cec73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.686240 11591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04180ace5d4548dfbcd226bf229d5c26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.689616 11587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e52ec7d802a4e90bccef777f77a2a1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.695189 11581 tablet_service.cc:1467] Processing CreateTablet for tablet 90f62593b6f844fc8a41b9be240101fd (DEFAULT_TABLE table=default.Table_904e49ad16464cdbac83d37d3cdf115b [id=b62d16bd5d214bb5bae010b4eae39c19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.697153 11581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90f62593b6f844fc8a41b9be240101fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.716006 11580 tablet_service.cc:1467] Processing CreateTablet for tablet 9feaf1ed8ff34928b716fd0b4e41cdde (DEFAULT_TABLE table=default.Table_783e2f568d774b5eb908860af5fa4166 [id=d5913e299402470898908b32cdba262a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:26.717754 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9feaf1ed8ff34928b716fd0b4e41cdde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:26.749127 11842 raft_consensus.cc:482] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:26.749806 11842 raft_consensus.cc:504] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:26.764811 11842 leader_election.cc:288] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:26.789974 11878 consensus_queue.cc:1035] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 07:35:26.801993 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f31a069dc888487ebbbb1cb91e886ad0" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:26.803814 11119 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_3231dafd497f4c8e8a13703613affde3" } new_table_name: "default.Table_ba602ce39d464e51bd0faa02b98f4363" modify_external_catalogs: true
I20260926 07:35:26.808756 11878 raft_consensus.cc:482] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:26.809231 11878 raft_consensus.cc:504] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:26.813191 11878 leader_election.cc:288] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:26.813318 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a255ef54f7e410ca8a779e5d7a118ca" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:26.814194 11466 raft_consensus.cc:2454] T 9a255ef54f7e410ca8a779e5d7a118ca P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 0.
I20260926 07:35:26.815526 11247 leader_election.cc:302] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:26.816625 11878 raft_consensus.cc:2781] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:26.817132 11878 raft_consensus.cc:482] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:26.817670 11878 raft_consensus.cc:3037] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:26.824301 11464 raft_consensus.cc:2454] T 14e5da3711fe46c4bb46a7a60f41a581 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 1.
I20260926 07:35:26.825577 11247 leader_election.cc:302] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 6070e9e3d1234a9d83e49431416afade
I20260926 07:35:26.844417 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a255ef54f7e410ca8a779e5d7a118ca" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:26.838068 11900 raft_consensus.cc:2781] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:26.869748 11900 raft_consensus.cc:686] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Becoming Leader. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Running, Role: LEADER
I20260926 07:35:26.870630 11900 consensus_queue.cc:237] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:26.901495 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f31a069dc888487ebbbb1cb91e886ad0" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:26.908196 11609 raft_consensus.cc:2379] T f31a069dc888487ebbbb1cb91e886ad0 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 22f3902d01584936b20437606cb86dd1 in current term 1: Already voted for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in this term.
W20260926 07:35:26.944813 11249 leader_election.cc:341] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:26.949595 11402 leader_election.cc:302] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 22f3902d01584936b20437606cb86dd1; no voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:26.952399 11871 ts_tablet_manager.cc:1905] T 8cc53f05536743b8a24ab2c88937f8f2 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:26.958494 11842 raft_consensus.cc:3037] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:26.806308 11120 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_8304ef6caf564378a58911a50bc8d9f7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:27.006795 11120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8304ef6caf564378a58911a50bc8d9f7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:27.020732 11116 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_b9f92e070aab45cdb8352c1d8a3d603c" } modify_external_catalogs: true
I20260926 07:35:27.021286 11116 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_b9f92e070aab45cdb8352c1d8a3d603c" } modify_external_catalogs: true
I20260926 07:35:27.024317 11115 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_00b32220835a4f1681581a6c9a426f17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:27.025823 11115 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_00b32220835a4f1681581a6c9a426f17 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:27.044422 11114 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_8c8e3eb07a9d4b18a589cf8940bbe83f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:27.046662 11114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8c8e3eb07a9d4b18a589cf8940bbe83f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:35:27.088239 11107 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.10.171.3:45127, user_credentials={real_user=slave}} blocked reactor thread for 78692.4us
W20260926 07:35:27.091741 11247 consensus_peers.cc:487] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e -> Peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Couldn't send request to peer 22f3902d01584936b20437606cb86dd1. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260926 07:35:27.100926 11316 consensus_meta.cc:287] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent flushing consensus metadata: real 0.678s user 0.012s sys 0.000s
I20260926 07:35:27.101440 11316 raft_consensus.cc:2454] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
W20260926 07:35:27.102926 11104 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.171.61:38823, user_credentials={real_user=slave}} blocked reactor thread for 98108.9us
W20260926 07:35:27.103544 11107 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3e28ca159d8349748812825ac1947c51 (table default.Table_9444fa8f3e7e4c7e9a245b34db0fb9f1 [id=cc4663f9d1734d9cb4cc4e1f061310a3], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3e28ca159d8349748812825ac1947c51 consensus state has no leader
I20260926 07:35:27.106248 11752 ts_tablet_manager.cc:1918] T aa02661e7531402687f03e86dbc19418 P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:27.106776 11752 log.cc:1196] T aa02661e7531402687f03e86dbc19418 P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/aa02661e7531402687f03e86dbc19418
I20260926 07:35:27.107601 11752 ts_tablet_manager.cc:1939] T aa02661e7531402687f03e86dbc19418 P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:27.162528 11857 consensus_queue.cc:1035] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:27.162832 11401 leader_election.cc:302] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 6070e9e3d1234a9d83e49431416afade
I20260926 07:35:27.173300 11871 ts_tablet_manager.cc:1918] T 8cc53f05536743b8a24ab2c88937f8f2 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:27.173678 11871 log.cc:1196] T 8cc53f05536743b8a24ab2c88937f8f2 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/8cc53f05536743b8a24ab2c88937f8f2
I20260926 07:35:27.174350 11871 ts_tablet_manager.cc:1939] T 8cc53f05536743b8a24ab2c88937f8f2 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:27.185397 11911 raft_consensus.cc:2781] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
W20260926 07:35:27.188205 11288 consensus_meta.cc:287] T 4239f5d48512403bb711ece401f0756b P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent flushing consensus metadata: real 0.561s user 0.005s sys 0.011s
I20260926 07:35:27.192143 11754 tablet_bootstrap.cc:492] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:27.192744 11754 ts_tablet_manager.cc:1397] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.792s user 0.012s sys 0.006s
I20260926 07:35:27.195596 11754 raft_consensus.cc:348] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.196622 11754 raft_consensus.cc:374] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:27.196931 11754 raft_consensus.cc:729] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:27.197942 11754 consensus_queue.cc:260] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.210647 11911 raft_consensus.cc:686] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:27.211627 11911 consensus_queue.cc:237] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.220739 11907 raft_consensus.cc:482] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:27.221160 11907 raft_consensus.cc:504] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.222986 11907 leader_election.cc:288] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:27.224414 11464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0bdf6a4656e4217b0a7013fddc4cca8" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:27.224814 11754 ts_tablet_manager.cc:1428] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.032s user 0.007s sys 0.000s
I20260926 07:35:27.225102 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0bdf6a4656e4217b0a7013fddc4cca8" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:27.225615 11754 tablet_bootstrap.cc:492] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:27.225656 11315 raft_consensus.cc:2454] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 0.
W20260926 07:35:27.225786 11545 leader_election.cc:341] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:27.226907 11546 leader_election.cc:302] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1
I20260926 07:35:27.228699 11907 raft_consensus.cc:2781] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:27.229056 11907 raft_consensus.cc:482] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:27.229315 11907 raft_consensus.cc:3037] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
W20260926 07:35:27.238457 11112 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 248ab84a1f284c2bbddf62f00b485e13 has no consensus state
W20260926 07:35:27.239770 11113 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.3:53335 (request call id 25) took 1011 ms (client timeout 1000 ms). Trace:
W20260926 07:35:27.240099 11113 rpcz_store.cc:269] 0926 07:35:26.228173 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.228336 (+ 163us) service_pool.cc:227] Handling call
0926 07:35:27.239702 (+1011366us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":2}
I20260926 07:35:27.243158 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 70) took 1165 ms. Trace:
I20260926 07:35:27.243505 11871 rpcz_store.cc:276] 0926 07:35:26.077355 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.081266 (+ 3911us) service_pool.cc:227] Handling call
0926 07:35:26.358036 (+276770us) ts_tablet_manager.cc:1145] Deleting tablet 8cc53f05536743b8a24ab2c88937f8f2
0926 07:35:26.358077 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:27.094292 (+736215us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:27.161899 (+ 67607us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:27.243102 (+ 81203us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":276203}
I20260926 07:35:27.245726 11871 tablet_replica.cc:330] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:27.247412 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 8cc53f05536743b8a24ab2c88937f8f2 (table default.Table_836a28372ec4439d98b1583a9b50d2c1 [id=800a96cbd0b34639bbe7ad317a640053]) successfully deleted
I20260926 07:35:27.246611 11871 raft_consensus.cc:2229] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:27.249420 11871 pending_rounds.cc:62] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade: Trying to abort 1 pending ops.
I20260926 07:35:27.249740 11871 pending_rounds.cc:68] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333511681397276672 op_type: NO_OP noop_request { }
I20260926 07:35:27.250255 11871 raft_consensus.cc:2866] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 07:35:27.250703 11871 raft_consensus.cc:2258] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:27.252866 11871 ts_tablet_manager.cc:1905] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:27.266880 11878 raft_consensus.cc:504] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:27.269536 11464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a255ef54f7e410ca8a779e5d7a118ca" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:27.270107 11464 raft_consensus.cc:3037] T 9a255ef54f7e410ca8a779e5d7a118ca P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:27.285475 11754 tablet_bootstrap.cc:654] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:27.315263 11904 raft_consensus.cc:482] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:27.315735 11904 raft_consensus.cc:504] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.317649 11904 leader_election.cc:288] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:27.320398 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e5da3711fe46c4bb46a7a60f41a581" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:27.321139 11465 raft_consensus.cc:2379] T 14e5da3711fe46c4bb46a7a60f41a581 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6070e9e3d1234a9d83e49431416afade in current term 1: Already voted for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in this term.
I20260926 07:35:27.333921 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 56) took 1665 ms. Trace:
I20260926 07:35:27.336371 11754 tablet_bootstrap.cc:492] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:27.336818 11754 ts_tablet_manager.cc:1397] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.111s user 0.009s sys 0.005s
W20260926 07:35:27.341774 11756 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.929s user 0.068s sys 0.128s
W20260926 07:35:27.342042 11756 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.929s user 0.068s sys 0.128s
I20260926 07:35:27.344027 11756 tablet_bootstrap.cc:492] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:27.343523 11907 raft_consensus.cc:504] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.344461 11756 ts_tablet_manager.cc:1397] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 1.010s user 0.078s sys 0.128s
I20260926 07:35:27.344600 11316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e5da3711fe46c4bb46a7a60f41a581" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:27.345628 11907 leader_election.cc:288] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:27.353118 11756 raft_consensus.cc:348] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.354229 11756 raft_consensus.cc:374] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:27.354509 11756 raft_consensus.cc:729] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:27.355074 11756 consensus_queue.cc:260] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.356884 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0bdf6a4656e4217b0a7013fddc4cca8" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:27.357378 11315 raft_consensus.cc:3037] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:27.365101 11756 ts_tablet_manager.cc:1428] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20260926 07:35:27.366302 11756 tablet_bootstrap.cc:492] T 1620b507f07c446c8c1811895a097409 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:27.368243 11752 rpcz_store.cc:276] 0926 07:35:25.445042 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:25.445467 (+ 425us) service_pool.cc:227] Handling call
0926 07:35:26.564686 (+1119219us) ts_tablet_manager.cc:1145] Deleting tablet aa02661e7531402687f03e86dbc19418
0926 07:35:26.564820 (+ 134us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:26.779044 (+214224us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:27.106046 (+327002us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:27.110182 (+ 4136us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1118584}
I20260926 07:35:27.369393 11752 tablet_replica.cc:330] T afa901cdc6a84220bf36ee669a955b72 P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
W20260926 07:35:27.333266 11746 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.694s user 0.041s sys 0.094s
I20260926 07:35:27.349023 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0bdf6a4656e4217b0a7013fddc4cca8" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:27.349890 11464 raft_consensus.cc:2454] T 9a255ef54f7e410ca8a779e5d7a118ca P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 1.
I20260926 07:35:27.370108 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet aa02661e7531402687f03e86dbc19418 (table default.Table_17e4f8bb2ea7464fbc563d08108d8325 [id=d91b2ce98c4f439e890cbf47ac6c2d11]) successfully deleted
I20260926 07:35:27.376417 11754 raft_consensus.cc:348] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.377043 11754 raft_consensus.cc:374] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260926 07:35:27.377753 11545 leader_election.cc:341] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:27.377275 11754 raft_consensus.cc:729] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:27.380303 11754 consensus_queue.cc:260] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
W20260926 07:35:27.382438 11746 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.743s user 0.042s sys 0.094s
I20260926 07:35:27.386379 11746 tablet_bootstrap.cc:492] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:27.386773 11746 ts_tablet_manager.cc:1397] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.792s user 0.049s sys 0.095s
I20260926 07:35:27.389940 11754 ts_tablet_manager.cc:1428] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.053s user 0.006s sys 0.000s
I20260926 07:35:27.390695 11754 tablet_bootstrap.cc:492] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:27.402280 11756 tablet_bootstrap.cc:654] T 1620b507f07c446c8c1811895a097409 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:27.350920 11445 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:40244 (request call id 71) took 1010 ms. Trace:
I20260926 07:35:27.407497 11247 leader_election.cc:302] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:27.412719 11754 tablet_bootstrap.cc:654] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:27.413187 11104 consensus_peers.cc:487] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d -> Peer d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629): Couldn't send request to peer d6ef5b900d0f4e73a358f415efa97249. Status: Network error: Client connection negotiation failed: client connection to 127.10.171.62:37629: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260926 07:35:27.408339 11901 raft_consensus.cc:2781] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:27.416788 11901 raft_consensus.cc:686] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Becoming Leader. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Running, Role: LEADER
I20260926 07:35:27.419222 11752 raft_consensus.cc:2229] T afa901cdc6a84220bf36ee669a955b72 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:27.419695 11752 raft_consensus.cc:2258] T afa901cdc6a84220bf36ee669a955b72 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:27.422016 11752 ts_tablet_manager.cc:1905] T afa901cdc6a84220bf36ee669a955b72 P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:27.422039 11878 leader_election.cc:288] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Requested vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:27.424556 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a255ef54f7e410ca8a779e5d7a118ca" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:27.425952 11546 leader_election.cc:302] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:27.427369 11907 raft_consensus.cc:3037] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:27.406942 11445 rpcz_store.cc:276] 0926 07:35:26.340659 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.340841 (+ 182us) service_pool.cc:227] Handling call
0926 07:35:26.342203 (+ 1362us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0926 07:35:26.342272 (+ 69us) ts_tablet_manager.cc:676] Creating new metadata...
0926 07:35:27.224273 (+882001us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:27.350494 (+126221us) tablet_replica.cc:205] Creating consensus instance
0926 07:35:27.350871 (+ 377us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260926 07:35:27.418252 11901 consensus_queue.cc:237] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:27.433413 11249 leader_election.cc:341] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:27.439446 11111 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:44836 (request call id 153) took 1034 ms. Trace:
I20260926 07:35:27.439769 11111 rpcz_store.cc:276] 0926 07:35:26.405193 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.415945 (+ 10752us) service_pool.cc:227] Handling call
0926 07:35:26.418372 (+ 2427us) catalog_manager.cc:2075] Acquired catalog manager lock
0926 07:35:26.423356 (+ 4984us) catalog_manager.cc:2121] Created new table and tablet info
0926 07:35:27.438821 (+1015465us) catalog_manager.cc:2181] Wrote table and tablets to system table
0926 07:35:27.439121 (+ 300us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0926 07:35:27.439389 (+ 268us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:26.464984 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:26.465514 (+ 530us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:26.465547 (+ 33us) write_op.cc:435] Acquired schema lock
0926 07:35:26.465566 (+ 19us) tablet.cc:660] Decoding operations
0926 07:35:26.465923 (+ 357us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:26.465970 (+ 47us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:26.466003 (+ 33us) tablet.cc:683] Acquiring locks for 2 operations
0926 07:35:26.466257 (+ 254us) tablet.cc:699] Row locks acquired
0926 07:35:26.479793 (+ 13536us) write_op.cc:260] PREPARE: finished
0926 07:35:26.479930 (+ 137us) write_op.cc:270] Start()
0926 07:35:26.480026 (+ 96us) write_op.cc:276] Timestamp: P: 1790408126479854 usec, L: 0
0926 07:35:26.480044 (+ 18us) op_driver.cc:331] REPLICATION: starting
0926 07:35:26.484273 (+ 4229us) log.cc:842] Serialized 433 byte log entry
0926 07:35:27.434502 (+950229us) op_driver.cc:444] REPLICATION: finished
0926 07:35:27.437010 (+ 2508us) write_op.cc:301] APPLY: starting
0926 07:35:27.437055 (+ 45us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:27.437140 (+ 85us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:27.437216 (+ 76us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:27.437566 (+ 350us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:27.437687 (+ 121us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 07:35:27.437735 (+ 48us) write_op.cc:312] APPLY: finished
0926 07:35:27.437871 (+ 136us) log.cc:842] Serialized 29 byte log entry
0926 07:35:27.438001 (+ 130us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:27.438059 (+ 58us) write_op.cc:454] Released schema lock
0926 07:35:27.438085 (+ 26us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:27.438221 (+ 136us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:27.438242 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2391,"apply.run_cpu_time_us":1351,"apply.run_wall_time_us":1351,"num_ops":2,"prepare.queue_time_us":20235,"prepare.run_cpu_time_us":2523,"prepare.run_wall_time_us":23437,"raft.queue_time_us":3625,"raft.run_cpu_time_us":23,"raft.run_wall_time_us":26,"replication_time_us":954398,"threads_started":1}]]}
I20260926 07:35:27.447587 11907 raft_consensus.cc:2726] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:27.451632 11315 raft_consensus.cc:2454] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 1.
I20260926 07:35:27.452739 11546 leader_election.cc:302] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1
I20260926 07:35:27.452913 11842 raft_consensus.cc:2726] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:27.453758 11907 raft_consensus.cc:2781] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:27.454414 11907 raft_consensus.cc:686] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Becoming Leader. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Running, Role: LEADER
I20260926 07:35:27.455161 11907 consensus_queue.cc:237] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.449617 11292 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:52070 (request call id 71) took 1117 ms. Trace:
I20260926 07:35:27.461777 11292 rpcz_store.cc:276] 0926 07:35:26.332040 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.332377 (+ 337us) service_pool.cc:227] Handling call
0926 07:35:26.333642 (+ 1265us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0926 07:35:26.333722 (+ 80us) ts_tablet_manager.cc:676] Creating new metadata...
0926 07:35:27.155825 (+822103us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:27.372052 (+216227us) tablet_replica.cc:205] Creating consensus instance
0926 07:35:27.449553 (+ 77501us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260926 07:35:27.464006 11119 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cef2e44be4084fc78e61cf822659b4ea has no consensus state
I20260926 07:35:27.466699 11119 master_service.cc:946] ReplaceTablet: received request to replace tablet c36c759963cb4915afdca7d12109cc6e from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:27.469754 11120 master_service.cc:946] ReplaceTablet: received request to replace tablet 04be706330924282961054c5835cec73 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:27.470371 11295 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:52070 (request call id 77) took 1051 ms. Trace:
I20260926 07:35:27.470745 11295 rpcz_store.cc:276] 0926 07:35:26.419352 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.424990 (+ 5638us) service_pool.cc:227] Handling call
0926 07:35:26.662485 (+237495us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0926 07:35:26.662625 (+ 140us) ts_tablet_manager.cc:676] Creating new metadata...
0926 07:35:27.228710 (+566085us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:27.465383 (+236673us) tablet_replica.cc:205] Creating consensus instance
0926 07:35:27.470299 (+ 4916us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260926 07:35:27.476635 11437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:40244 (request call id 80) took 1041 ms. Trace:
I20260926 07:35:27.478688 11116 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet c36c759963cb4915afdca7d12109cc6e
I20260926 07:35:27.480336 11437 rpcz_store.cc:276] 0926 07:35:26.435148 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.435300 (+ 152us) service_pool.cc:227] Handling call
0926 07:35:26.436900 (+ 1600us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0926 07:35:26.437004 (+ 104us) ts_tablet_manager.cc:676] Creating new metadata...
0926 07:35:27.212278 (+775274us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:27.476092 (+263814us) tablet_replica.cc:205] Creating consensus instance
0926 07:35:27.476578 (+ 486us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260926 07:35:27.480785 11116 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_a84cdaff0c794776a40dc68d90b7ce79" } new_table_name: "default.Table_1b1901dc5bbb4288be18304dd4a9b113" modify_external_catalogs: true
I20260926 07:35:27.486372 11901 consensus_queue.cc:1035] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:27.488408 11436 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:40244 (request call id 81) took 1050 ms. Trace:
I20260926 07:35:27.490361 11589 tablet_service.cc:1514] Processing DeleteTablet for tablet c36c759963cb4915afdca7d12109cc6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:27 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:27.494812 11756 tablet_bootstrap.cc:492] T 1620b507f07c446c8c1811895a097409 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:27.495167 11756 ts_tablet_manager.cc:1397] T 1620b507f07c446c8c1811895a097409 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.129s user 0.012s sys 0.005s
I20260926 07:35:27.497124 11756 raft_consensus.cc:348] T 1620b507f07c446c8c1811895a097409 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.497746 11756 raft_consensus.cc:374] T 1620b507f07c446c8c1811895a097409 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:27.500952 11754 tablet_bootstrap.cc:492] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:27.501332 11754 ts_tablet_manager.cc:1397] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.111s user 0.008s sys 0.000s
I20260926 07:35:27.489584 11446 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:40244 (request call id 76) took 1082 ms. Trace:
I20260926 07:35:27.501689 11446 rpcz_store.cc:276] 0926 07:35:26.406871 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.407175 (+ 304us) service_pool.cc:227] Handling call
0926 07:35:26.415644 (+ 8469us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0926 07:35:26.415717 (+ 73us) ts_tablet_manager.cc:676] Creating new metadata...
0926 07:35:27.205287 (+789570us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:27.489092 (+283805us) tablet_replica.cc:205] Creating consensus instance
0926 07:35:27.489534 (+ 442us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":20}
I20260926 07:35:27.490109 11439 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:40244 (request call id 77) took 1083 ms. Trace:
I20260926 07:35:27.502625 11439 rpcz_store.cc:276] 0926 07:35:26.407220 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.412960 (+ 5740us) service_pool.cc:227] Handling call
0926 07:35:26.414611 (+ 1651us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0926 07:35:26.414685 (+ 74us) ts_tablet_manager.cc:676] Creating new metadata...
0926 07:35:27.204164 (+789479us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:27.489148 (+284984us) tablet_replica.cc:205] Creating consensus instance
0926 07:35:27.490063 (+ 915us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260926 07:35:27.491266 11444 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:40244 (request call id 78) took 1083 ms. Trace:
I20260926 07:35:27.497128 11435 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:40244 (request call id 82) took 1053 ms. Trace:
I20260926 07:35:27.503727 11435 rpcz_store.cc:276] 0926 07:35:26.443345 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.444999 (+ 1654us) service_pool.cc:227] Handling call
0926 07:35:26.447965 (+ 2966us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0926 07:35:26.448793 (+ 828us) ts_tablet_manager.cc:676] Creating new metadata...
0926 07:35:27.163883 (+715090us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:27.496709 (+332826us) tablet_replica.cc:205] Creating consensus instance
0926 07:35:27.497077 (+ 368us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260926 07:35:27.503749 11444 rpcz_store.cc:276] 0926 07:35:26.408009 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.408245 (+ 236us) service_pool.cc:227] Handling call
0926 07:35:26.409777 (+ 1532us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0926 07:35:26.409872 (+ 95us) ts_tablet_manager.cc:676] Creating new metadata...
0926 07:35:27.220889 (+811017us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:27.490872 (+269983us) tablet_replica.cc:205] Creating consensus instance
0926 07:35:27.491215 (+ 343us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260926 07:35:27.497982 11756 raft_consensus.cc:729] T 1620b507f07c446c8c1811895a097409 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:27.509131 11756 consensus_queue.cc:260] T 1620b507f07c446c8c1811895a097409 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.510074 11857 raft_consensus.cc:482] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:27.510485 11857 raft_consensus.cc:504] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.512235 11857 leader_election.cc:288] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:27.514516 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4826318c83524f41840e770f33bd3696" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
W20260926 07:35:27.516731 11249 leader_election.cc:341] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:27.520711 11754 raft_consensus.cc:348] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.521382 11754 raft_consensus.cc:374] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:27.521636 11754 raft_consensus.cc:729] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:27.504912 11436 rpcz_store.cc:276] 0926 07:35:26.440298 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.440539 (+ 241us) service_pool.cc:227] Handling call
0926 07:35:26.442501 (+ 1962us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0926 07:35:26.442626 (+ 125us) ts_tablet_manager.cc:676] Creating new metadata...
0926 07:35:27.196260 (+753634us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:27.487935 (+291675us) tablet_replica.cc:205] Creating consensus instance
0926 07:35:27.488342 (+ 407us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260926 07:35:27.528893 11756 ts_tablet_manager.cc:1428] T 1620b507f07c446c8c1811895a097409 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.033s user 0.006s sys 0.001s
I20260926 07:35:27.533054 11753 ts_tablet_manager.cc:1918] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:27.532900 11754 consensus_queue.cc:260] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.543269 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4826318c83524f41840e770f33bd3696" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:27.545195 11754 ts_tablet_manager.cc:1428] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.044s user 0.003s sys 0.003s
I20260926 07:35:27.491207 11292 tablet_service.cc:1514] Processing DeleteTablet for tablet c36c759963cb4915afdca7d12109cc6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:27 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:27.552337 11753 log.cc:1196] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/345d14fc9eea46ad9b1155d538118ee5
I20260926 07:35:27.553171 11753 ts_tablet_manager.cc:1939] T 345d14fc9eea46ad9b1155d538118ee5 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:27.546833 11436 tablet_service.cc:1514] Processing DeleteTablet for tablet c36c759963cb4915afdca7d12109cc6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:27 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:27.535502 11746 raft_consensus.cc:348] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.557314 11746 raft_consensus.cc:374] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:27.560652 11754 tablet_bootstrap.cc:492] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:27.574723 11754 tablet_bootstrap.cc:654] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:27.582847 11114 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:44836 (request call id 154) took 1041 ms. Trace:
I20260926 07:35:27.583184 11114 rpcz_store.cc:276] 0926 07:35:26.541749 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:27.005155 (+463406us) service_pool.cc:227] Handling call
0926 07:35:27.018960 (+ 13805us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b5400003110. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb809295f11 00007fb809419851 00007fb809419d92 00007fb8123117c9 00007fb8122a9c36 00007fb81227ac28 00007fb812376156 00007fb80feb3034 00007fb80feb2fc2 00007fb80feb2ef1 00007fb80feb2e6c
0926 07:35:27.043544 (+ 24584us) spinlock_profiling.cc:243] Waited 9.58 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812376156 00007fb80feb3034 00007fb80feb2fc2 00007fb80feb2ef1 00007fb80feb2e6c 00007fb80feb2122 00007fb80e15594c 00007fb80e154d86
0926 07:35:27.047216 (+ 3672us) catalog_manager.cc:2075] Acquired catalog manager lock
0926 07:35:27.050526 (+ 3310us) catalog_manager.cc:2121] Created new table and tablet info
0926 07:35:27.582130 (+531604us) catalog_manager.cc:2181] Wrote table and tablets to system table
0926 07:35:27.582508 (+ 378us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0926 07:35:27.582791 (+ 283us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:27.569206 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:27.570032 (+ 826us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:27.570056 (+ 24us) write_op.cc:435] Acquired schema lock
0926 07:35:27.570071 (+ 15us) tablet.cc:660] Decoding operations
0926 07:35:27.570344 (+ 273us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:27.570395 (+ 51us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:27.570418 (+ 23us) tablet.cc:683] Acquiring locks for 2 operations
0926 07:35:27.570617 (+ 199us) tablet.cc:699] Row locks acquired
0926 07:35:27.570633 (+ 16us) write_op.cc:260] PREPARE: finished
0926 07:35:27.570732 (+ 99us) write_op.cc:270] Start()
0926 07:35:27.570805 (+ 73us) write_op.cc:276] Timestamp: P: 1790408127570664 usec, L: 0
0926 07:35:27.570823 (+ 18us) op_driver.cc:331] REPLICATION: starting
0926 07:35:27.571277 (+ 454us) log.cc:842] Serialized 433 byte log entry
0926 07:35:27.579898 (+ 8621us) op_driver.cc:444] REPLICATION: finished
0926 07:35:27.580098 (+ 200us) write_op.cc:301] APPLY: starting
0926 07:35:27.580569 (+ 471us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:27.580687 (+ 118us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:27.580780 (+ 93us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:27.581144 (+ 364us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:27.581263 (+ 119us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 07:35:27.581303 (+ 40us) write_op.cc:312] APPLY: finished
0926 07:35:27.581471 (+ 168us) log.cc:842] Serialized 29 byte log entry
0926 07:35:27.581588 (+ 117us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:27.581648 (+ 60us) write_op.cc:454] Released schema lock
0926 07:35:27.581676 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:27.581840 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:27.581867 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":27559552,"child_traces":[["op",{"apply.queue_time_us":71,"apply.run_cpu_time_us":1522,"apply.run_wall_time_us":1898,"num_ops":2,"prepare.queue_time_us":509124,"raft.queue_time_us":58,"raft.run_cpu_time_us":549,"raft.run_wall_time_us":1172,"replication_time_us":9006}]]}
I20260926 07:35:27.596204 11746 raft_consensus.cc:729] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:27.596971 11746 consensus_queue.cc:260] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.599318 11746 ts_tablet_manager.cc:1428] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.212s user 0.005s sys 0.001s
W20260926 07:35:27.545517 11247 leader_election.cc:341] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:27.607770 11247 leader_election.cc:302] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade
W20260926 07:35:27.608819 11247 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.2:42379, user_credentials={real_user=slave}} blocked reactor thread for 63337.3us
I20260926 07:35:27.610163 11878 raft_consensus.cc:2726] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:27.487480 11290 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:52070 (request call id 78) took 1067 ms. Trace:
I20260926 07:35:27.534432 11756 tablet_bootstrap.cc:492] T e06ed286144941269b9a0c0c60bff0ce P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:27.611132 11290 rpcz_store.cc:276] 0926 07:35:26.419617 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.419729 (+ 112us) service_pool.cc:227] Handling call
0926 07:35:26.437786 (+ 18057us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0926 07:35:26.437943 (+ 157us) ts_tablet_manager.cc:676] Creating new metadata...
0926 07:35:27.370207 (+932264us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:27.487049 (+116842us) tablet_replica.cc:205] Creating consensus instance
0926 07:35:27.487424 (+ 375us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260926 07:35:27.614009 11901 consensus_queue.cc:1035] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:27.616434 11756 tablet_bootstrap.cc:654] T e06ed286144941269b9a0c0c60bff0ce P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:27.622371 11754 tablet_bootstrap.cc:492] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:27.622766 11754 ts_tablet_manager.cc:1397] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.062s user 0.009s sys 0.000s
W20260926 07:35:27.631676 11359 debug-util.cc:398] Leaking SignalData structure 0x7b08000d2bc0 after lost signal to thread 11223
W20260926 07:35:27.632875 11359 debug-util.cc:398] Leaking SignalData structure 0x7b08000fc340 after lost signal to thread 11362
W20260926 07:35:27.633190 11359 debug-util.cc:398] Leaking SignalData structure 0x7b08000fc300 after lost signal to thread 11883
I20260926 07:35:27.634860 11754 raft_consensus.cc:348] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.635519 11754 raft_consensus.cc:374] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:27.635758 11754 raft_consensus.cc:729] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:27.637720 11901 consensus_queue.cc:1035] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:27.642886 11753 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 58) took 2212 ms. Trace:
I20260926 07:35:27.643260 11753 rpcz_store.cc:276] 0926 07:35:25.430617 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:25.432711 (+ 2094us) service_pool.cc:227] Handling call
0926 07:35:26.563557 (+1130846us) ts_tablet_manager.cc:1145] Deleting tablet 345d14fc9eea46ad9b1155d538118ee5
0926 07:35:26.665117 (+101560us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:27.350551 (+685434us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:27.532860 (+182309us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:27.642826 (+109966us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1104380}
I20260926 07:35:27.645809 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet 345d14fc9eea46ad9b1155d538118ee5 (table default.Table_c1c4fe0ecbe840e59d8ba4f659432e0d [id=4c9f2ba8e0db427da572f73f02d4fa9e]) successfully deleted
I20260926 07:35:27.658733 11754 consensus_queue.cc:260] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.660727 11754 ts_tablet_manager.cc:1428] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.038s user 0.007s sys 0.000s
I20260926 07:35:27.661378 11756 tablet_bootstrap.cc:492] T e06ed286144941269b9a0c0c60bff0ce P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:27.661754 11756 ts_tablet_manager.cc:1397] T e06ed286144941269b9a0c0c60bff0ce P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.131s user 0.011s sys 0.000s
I20260926 07:35:27.663903 11756 raft_consensus.cc:348] T e06ed286144941269b9a0c0c60bff0ce P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.665048 11756 raft_consensus.cc:374] T e06ed286144941269b9a0c0c60bff0ce P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:27.665301 11756 raft_consensus.cc:729] T e06ed286144941269b9a0c0c60bff0ce P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:27.665832 11756 consensus_queue.cc:260] T e06ed286144941269b9a0c0c60bff0ce P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.667620 11756 ts_tablet_manager.cc:1428] T e06ed286144941269b9a0c0c60bff0ce P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260926 07:35:27.679164 11754 tablet_bootstrap.cc:492] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:27.681142 11756 tablet_bootstrap.cc:492] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:27.688663 11857 consensus_queue.cc:1035] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:27.700459 11756 tablet_bootstrap.cc:654] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:27.706570 11754 tablet_bootstrap.cc:654] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:27.708966 11878 consensus_queue.cc:1035] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:27.733392 11756 tablet_bootstrap.cc:492] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:27.733791 11756 ts_tablet_manager.cc:1397] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.053s user 0.002s sys 0.006s
I20260926 07:35:27.736056 11756 raft_consensus.cc:348] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.737047 11756 raft_consensus.cc:374] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:27.737293 11756 raft_consensus.cc:729] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:27.737871 11756 consensus_queue.cc:260] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.739722 11756 ts_tablet_manager.cc:1428] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260926 07:35:27.742609 11746 tablet_bootstrap.cc:492] T 4826318c83524f41840e770f33bd3696 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:27.746299 11756 tablet_bootstrap.cc:492] T 1e1cea4523a64428aac03ca0bdf708c7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:27.749651 11754 tablet_bootstrap.cc:492] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:27.751792 11756 tablet_bootstrap.cc:654] T 1e1cea4523a64428aac03ca0bdf708c7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:27.752260 11754 ts_tablet_manager.cc:1397] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.074s user 0.009s sys 0.001s
I20260926 07:35:27.754473 11754 raft_consensus.cc:348] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:27.756263 11114 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_d94c4858b83943018a693428b6e8a578"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:27.757890 11114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d94c4858b83943018a693428b6e8a578 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:27.758312 11754 raft_consensus.cc:374] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:27.758623 11754 raft_consensus.cc:729] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:27.760779 11115 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_03b976b95c404da1b2b14a6d71dd8a72" } new_table_name: "default.Table_56493b3efdaa43e684afcd3d4b5dffdd" modify_external_catalogs: true
I20260926 07:35:27.761610 11754 consensus_queue.cc:260] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:27.764103 11754 ts_tablet_manager.cc:1428] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.012s user 0.004s sys 0.001s
I20260926 07:35:27.771371 11754 tablet_bootstrap.cc:492] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:27.776507 11754 tablet_bootstrap.cc:654] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:27.793514 11756 tablet_bootstrap.cc:492] T 1e1cea4523a64428aac03ca0bdf708c7 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:27.793910 11756 ts_tablet_manager.cc:1397] T 1e1cea4523a64428aac03ca0bdf708c7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.048s user 0.009s sys 0.001s
I20260926 07:35:27.795962 11756 raft_consensus.cc:348] T 1e1cea4523a64428aac03ca0bdf708c7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:27.796588 11756 raft_consensus.cc:374] T 1e1cea4523a64428aac03ca0bdf708c7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:27.796821 11756 raft_consensus.cc:729] T 1e1cea4523a64428aac03ca0bdf708c7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
W20260926 07:35:23.074922 11740 debug-util.cc:398] Leaking SignalData structure 0x7b0800014860 after lost signal to thread 11716
W20260926 07:35:27.800215 11740 kernel_stack_watchdog.cc:198] Thread 11716 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 350ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260926 07:35:27.803197 11942 log.cc:924] Time spent T 067ae4ea43f54e88874d93b4393dae27 P 22f3902d01584936b20437606cb86dd1: Append to log took a long time: real 0.062s user 0.001s sys 0.000s
W20260926 07:35:21.725747 11742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:35:27.805892 11716 thread.cc:636] GCE (cloud detector) Time spent creating pthread: real 6.083s user 0.380s sys 0.950s
W20260926 07:35:27.806205 11716 thread.cc:602] GCE (cloud detector) Time spent starting thread: real 6.084s user 0.380s sys 0.950s
I20260926 07:35:27.805940 11756 consensus_queue.cc:260] T 1e1cea4523a64428aac03ca0bdf708c7 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:27.808211 11754 tablet_bootstrap.cc:492] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:27.808619 11754 ts_tablet_manager.cc:1397] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.037s user 0.013s sys 0.000s
I20260926 07:35:27.810994 11754 raft_consensus.cc:348] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.814683 11754 raft_consensus.cc:374] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:27.815026 11754 raft_consensus.cc:729] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:27.815855 11754 consensus_queue.cc:260] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.818178 11754 ts_tablet_manager.cc:1428] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260926 07:35:27.820009 11754 tablet_bootstrap.cc:492] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:27.820785 11756 ts_tablet_manager.cc:1428] T 1e1cea4523a64428aac03ca0bdf708c7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.027s user 0.005s sys 0.000s
I20260926 07:35:27.823856 11716 server_base.cc:1034] running on GCE node
W20260926 07:35:27.824566 11950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:35:27.826020 11716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:35:27.826622 11754 tablet_bootstrap.cc:654] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:27.838487 11754 tablet_bootstrap.cc:492] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:27.838985 11754 ts_tablet_manager.cc:1397] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.019s user 0.014s sys 0.001s
I20260926 07:35:27.841177 11754 raft_consensus.cc:348] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.841965 11754 raft_consensus.cc:374] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:27.842278 11754 raft_consensus.cc:729] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:27.843006 11754 consensus_queue.cc:260] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.845216 11754 ts_tablet_manager.cc:1428] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 07:35:27.846271 11754 tablet_bootstrap.cc:492] T 75d0b86635944407b83f5ddfa29fcc4d P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:27.847771 11716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:35:27.852216 11754 tablet_bootstrap.cc:654] T 75d0b86635944407b83f5ddfa29fcc4d P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:27.862864 11754 tablet_bootstrap.cc:492] T 75d0b86635944407b83f5ddfa29fcc4d P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:27.863332 11754 ts_tablet_manager.cc:1397] T 75d0b86635944407b83f5ddfa29fcc4d P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.017s user 0.016s sys 0.000s
I20260926 07:35:27.865311 11754 raft_consensus.cc:348] T 75d0b86635944407b83f5ddfa29fcc4d P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.866032 11754 raft_consensus.cc:374] T 75d0b86635944407b83f5ddfa29fcc4d P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:27.866325 11754 raft_consensus.cc:729] T 75d0b86635944407b83f5ddfa29fcc4d P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:27.866878 11716 hybrid_clock.cc:648] HybridClock initialized: now 1790408127866814 us; error 63 us; skew 500 ppm
I20260926 07:35:27.866982 11754 consensus_queue.cc:260] T 75d0b86635944407b83f5ddfa29fcc4d P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:27.868018 11716 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:35:27.872767 11754 ts_tablet_manager.cc:1428] T 75d0b86635944407b83f5ddfa29fcc4d P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260926 07:35:27.873567 11754 tablet_bootstrap.cc:492] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:27.876746 11746 tablet_bootstrap.cc:654] T 4826318c83524f41840e770f33bd3696 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:27.878183 11754 tablet_bootstrap.cc:654] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:27.897545 11753 tablet_replica.cc:330] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:27.898219 11753 raft_consensus.cc:2229] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:27.905689 11907 raft_consensus.cc:482] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:27.906095 11907 raft_consensus.cc:504] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:27.907793 11907 leader_election.cc:288] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:27.909080 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a255ef54f7e410ca8a779e5d7a118ca" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:27.909909 11716 webserver.cc:460] Webserver started at http://127.10.171.62:37637/ using document root <none> and password file <none>
I20260926 07:35:27.910607 11754 tablet_bootstrap.cc:492] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:27.911033 11754 ts_tablet_manager.cc:1397] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.038s user 0.019s sys 0.007s
I20260926 07:35:27.911803 11716 fs_manager.cc:359] Metadata directory not provided
I20260926 07:35:27.912186 11716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:35:27.914191 11754 raft_consensus.cc:348] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:27.914778 11754 raft_consensus.cc:374] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:27.915011 11754 raft_consensus.cc:729] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:27.915542 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a255ef54f7e410ca8a779e5d7a118ca" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:27.915580 11754 consensus_queue.cc:260] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:27.916285 11463 raft_consensus.cc:2379] T 9a255ef54f7e410ca8a779e5d7a118ca P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6070e9e3d1234a9d83e49431416afade in current term 1: Already voted for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in this term.
I20260926 07:35:27.917757 11754 ts_tablet_manager.cc:1428] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 07:35:27.918494 11754 tablet_bootstrap.cc:492] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:27.921517 11545 leader_election.cc:302] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:27.922493 11907 raft_consensus.cc:3037] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:27.922935 11754 tablet_bootstrap.cc:654] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:27.899145 11753 raft_consensus.cc:2258] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:27.929801 11871 ts_tablet_manager.cc:1918] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:27.930182 11871 log.cc:1196] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/7000bcd55a984d64ba9d7d50fe63822c
I20260926 07:35:27.930924 11871 ts_tablet_manager.cc:1939] T 7000bcd55a984d64ba9d7d50fe63822c P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:27.934106 11754 tablet_bootstrap.cc:492] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:27.934461 11754 ts_tablet_manager.cc:1397] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.004s
I20260926 07:35:27.935070 11753 ts_tablet_manager.cc:1905] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:27.936815 11756 tablet_bootstrap.cc:492] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:27.936888 11754 raft_consensus.cc:348] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:27.938588 11754 raft_consensus.cc:374] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:27.938823 11754 raft_consensus.cc:729] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:27.939428 11754 consensus_queue.cc:260] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:27.941437 11756 tablet_bootstrap.cc:654] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:27.941838 11754 ts_tablet_manager.cc:1428] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260926 07:35:27.942552 11754 tablet_bootstrap.cc:492] T a55e54d492214039b0294f954b5ef390 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:27.949749 11716 fs_manager.cc:709] Time spent opening directory manager: real 0.026s user 0.005s sys 0.001s
I20260926 07:35:27.958159 11756 tablet_bootstrap.cc:492] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:27.958549 11756 ts_tablet_manager.cc:1397] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.002s
I20260926 07:35:27.960873 11756 raft_consensus.cc:348] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:27.961510 11756 raft_consensus.cc:374] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:27.961748 11756 raft_consensus.cc:729] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:27.963119 11907 raft_consensus.cc:2726] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:27.964524 11754 tablet_bootstrap.cc:654] T a55e54d492214039b0294f954b5ef390 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:27.964382 11756 consensus_queue.cc:260] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
W20260926 07:35:27.967293 11116 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c81a59361a6e42b7962b3e027c4a087f consensus state has no leader
I20260926 07:35:27.977403 11756 ts_tablet_manager.cc:1428] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.019s user 0.005s sys 0.001s
I20260926 07:35:27.978185 11756 tablet_bootstrap.cc:492] T 509f607b089c44bc9c3f80ed59a68c95 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:27.986001 11754 tablet_bootstrap.cc:492] T a55e54d492214039b0294f954b5ef390 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:27.986480 11754 ts_tablet_manager.cc:1397] T a55e54d492214039b0294f954b5ef390 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.044s user 0.008s sys 0.005s
I20260926 07:35:27.988534 11754 raft_consensus.cc:348] T a55e54d492214039b0294f954b5ef390 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:27.989166 11754 raft_consensus.cc:374] T a55e54d492214039b0294f954b5ef390 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:27.990959 11960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:35:27.992475 11716 fs_manager.cc:725] Time spent opening block manager: real 0.040s user 0.004s sys 0.019s
I20260926 07:35:27.994515 11756 tablet_bootstrap.cc:654] T 509f607b089c44bc9c3f80ed59a68c95 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:27.992972 11716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/data,/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/wal
uuid: "d6ef5b900d0f4e73a358f415efa97249"
format_stamp: "Formatted at 2026-09-26 07:35:07 on dist-test-slave-3mmx"
I20260926 07:35:27.997247 11716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:35:28.006461 11754 raft_consensus.cc:729] T a55e54d492214039b0294f954b5ef390 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:28.006649 11904 raft_consensus.cc:482] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:28.007071 11904 raft_consensus.cc:504] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:28.007151 11754 consensus_queue.cc:260] T a55e54d492214039b0294f954b5ef390 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:28.009552 11904 leader_election.cc:288] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:28.009959 11904 raft_consensus.cc:482] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:28.010347 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06ed286144941269b9a0c0c60bff0ce" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:28.010315 11904 raft_consensus.cc:504] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:28.010686 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06ed286144941269b9a0c0c60bff0ce" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:28.010982 11315 raft_consensus.cc:2454] T e06ed286144941269b9a0c0c60bff0ce P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 0.
W20260926 07:35:28.011830 11545 leader_election.cc:341] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:28.012243 11904 leader_election.cc:288] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:28.012657 11546 leader_election.cc:302] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1
I20260926 07:35:28.014307 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4826318c83524f41840e770f33bd3696" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:28.014371 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4826318c83524f41840e770f33bd3696" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:28.014840 11315 raft_consensus.cc:2454] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 0.
W20260926 07:35:28.015497 11545 leader_election.cc:341] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:28.015997 11546 leader_election.cc:302] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1
I20260926 07:35:28.016026 11907 raft_consensus.cc:2781] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:28.017400 11907 raft_consensus.cc:482] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:28.017795 11907 raft_consensus.cc:3037] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:28.023604 11904 raft_consensus.cc:2781] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:28.023911 11904 raft_consensus.cc:482] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:28.024230 11904 raft_consensus.cc:3037] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:28.034353 11589 tablet_service.cc:1467] Processing CreateTablet for tablet c092aff8f2a041ccb09bc4b48e0f359c (DEFAULT_TABLE table=default.Table_4c7ed90b32024fb888d7aef0e188ea80 [id=4e53b36526114bcabd8920c83fa2f6d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.035406 11589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c092aff8f2a041ccb09bc4b48e0f359c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.037801 11436 tablet_service.cc:1467] Processing CreateTablet for tablet c092aff8f2a041ccb09bc4b48e0f359c (DEFAULT_TABLE table=default.Table_4c7ed90b32024fb888d7aef0e188ea80 [id=4e53b36526114bcabd8920c83fa2f6d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.037946 11290 tablet_service.cc:1467] Processing CreateTablet for tablet c092aff8f2a041ccb09bc4b48e0f359c (DEFAULT_TABLE table=default.Table_4c7ed90b32024fb888d7aef0e188ea80 [id=4e53b36526114bcabd8920c83fa2f6d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.046446 11445 tablet_service.cc:1467] Processing CreateTablet for tablet 3f581510fe624127a4d27d302f5be614 (DEFAULT_TABLE table=default.Table_c111fc58a37e47878582962cca421abe [id=f1205638cbaa44c18698320192e5d22c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.047178 11583 tablet_service.cc:1467] Processing CreateTablet for tablet 3f581510fe624127a4d27d302f5be614 (DEFAULT_TABLE table=default.Table_c111fc58a37e47878582962cca421abe [id=f1205638cbaa44c18698320192e5d22c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.048278 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f581510fe624127a4d27d302f5be614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.032023 11111 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet a0bdf6a4656e4217b0a7013fddc4cca8 (table default.Table_c7fb27daeba04d90bc0cf0c7ac07fe56 [id=3ce63f9fb5ad4b82ba2fbbf2e3678271]). Expected version 1 got 0
I20260926 07:35:28.057039 11608 raft_consensus.cc:3037] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:28.059155 11292 tablet_service.cc:1467] Processing CreateTablet for tablet 3f581510fe624127a4d27d302f5be614 (DEFAULT_TABLE table=default.Table_c111fc58a37e47878582962cca421abe [id=f1205638cbaa44c18698320192e5d22c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.060918 11752 ts_tablet_manager.cc:1918] T afa901cdc6a84220bf36ee669a955b72 P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:28.060918 11753 ts_tablet_manager.cc:1918] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:28.061239 11753 log.cc:1196] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/aa02661e7531402687f03e86dbc19418
I20260926 07:35:28.061239 11752 log.cc:1196] T afa901cdc6a84220bf36ee669a955b72 P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/afa901cdc6a84220bf36ee669a955b72
I20260926 07:35:28.061851 11752 ts_tablet_manager.cc:1939] T afa901cdc6a84220bf36ee669a955b72 P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:28.061914 11753 ts_tablet_manager.cc:1939] T aa02661e7531402687f03e86dbc19418 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:28.062469 11907 raft_consensus.cc:504] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:28.063033 11292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f581510fe624127a4d27d302f5be614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.064744 11290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c092aff8f2a041ccb09bc4b48e0f359c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.064805 11907 leader_election.cc:288] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:28.066051 11436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c092aff8f2a041ccb09bc4b48e0f359c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.066956 11901 raft_consensus.cc:482] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:28.067378 11901 raft_consensus.cc:504] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:28.069239 11901 leader_election.cc:288] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:28.069541 11753 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 59) took 2609 ms. Trace:
I20260926 07:35:28.069819 11753 rpcz_store.cc:276] 0926 07:35:25.459877 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:25.460916 (+ 1039us) service_pool.cc:227] Handling call
0926 07:35:27.802424 (+2341508us) ts_tablet_manager.cc:1145] Deleting tablet aa02661e7531402687f03e86dbc19418
0926 07:35:27.890086 (+ 87662us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:27.956355 (+ 66269us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:28.060713 (+104358us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:28.069490 (+ 8777us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2340798}
I20260926 07:35:28.070641 11907 raft_consensus.cc:482] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:28.070797 11753 tablet_replica.cc:330] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:28.071148 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75d0b86635944407b83f5ddfa29fcc4d" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:28.071025 11907 raft_consensus.cc:504] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:28.071846 11753 raft_consensus.cc:2229] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Raft consensus shutting down.
W20260926 07:35:28.072556 11247 leader_election.cc:341] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:28.072943 11907 leader_election.cc:288] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:28.073884 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 71) took 1996 ms. Trace:
I20260926 07:35:28.074568 11871 rpcz_store.cc:276] 0926 07:35:26.077637 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.081928 (+ 4291us) service_pool.cc:227] Handling call
0926 07:35:27.245588 (+1163660us) ts_tablet_manager.cc:1145] Deleting tablet 7000bcd55a984d64ba9d7d50fe63822c
0926 07:35:27.245628 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:27.524441 (+278813us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:27.929565 (+405124us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:28.073733 (+144168us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1163116}
I20260926 07:35:28.076483 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet aa02661e7531402687f03e86dbc19418 (table default.Table_17e4f8bb2ea7464fbc563d08108d8325 [id=d91b2ce98c4f439e890cbf47ac6c2d11]) successfully deleted
I20260926 07:35:28.080329 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75d0b86635944407b83f5ddfa29fcc4d" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
W20260926 07:35:28.080715 11115 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0eae53ce278749208492815b918582f3 consensus state has no leader
I20260926 07:35:28.080922 11609 raft_consensus.cc:2454] T 75d0b86635944407b83f5ddfa29fcc4d P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 0.
I20260926 07:35:28.081753 11115 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_38183654ed414223a1467d57fc2d7d29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:28.082459 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06ed286144941269b9a0c0c60bff0ce" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:28.083016 11315 raft_consensus.cc:3037] T e06ed286144941269b9a0c0c60bff0ce P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:28.081748 11114 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_686ba040299440d0af09a9fe3501a040" } modify_external_catalogs: true
I20260926 07:35:28.083369 11114 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_686ba040299440d0af09a9fe3501a040" } modify_external_catalogs: true
I20260926 07:35:28.083429 11444 tablet_service.cc:1467] Processing CreateTablet for tablet dbd4f14abfea4799b75c48531412fc20 (DEFAULT_TABLE table=default.Table_e08de921f78841ccb011f4082de0fbe0 [id=e84ecb294af54e66984f108b907951ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.085091 11753 raft_consensus.cc:2258] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:28.087561 11295 tablet_service.cc:1467] Processing CreateTablet for tablet dbd4f14abfea4799b75c48531412fc20 (DEFAULT_TABLE table=default.Table_e08de921f78841ccb011f4082de0fbe0 [id=e84ecb294af54e66984f108b907951ab]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 07:35:28.083422 11115 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_38183654ed414223a1467d57fc2d7d29 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:28.088572 11295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd4f14abfea4799b75c48531412fc20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.092514 11316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1620b507f07c446c8c1811895a097409" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:28.093088 11316 raft_consensus.cc:2454] T 1620b507f07c446c8c1811895a097409 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 0.
I20260926 07:35:28.093283 11871 tablet_replica.cc:330] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:28.094182 11871 raft_consensus.cc:2229] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:28.094658 11871 raft_consensus.cc:2258] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:28.096248 11588 tablet_service.cc:1467] Processing CreateTablet for tablet dbd4f14abfea4799b75c48531412fc20 (DEFAULT_TABLE table=default.Table_e08de921f78841ccb011f4082de0fbe0 [id=e84ecb294af54e66984f108b907951ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.096493 11464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1620b507f07c446c8c1811895a097409" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:28.097034 11464 raft_consensus.cc:2454] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 0.
I20260926 07:35:28.097277 11588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd4f14abfea4799b75c48531412fc20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.099910 11871 ts_tablet_manager.cc:1905] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:28.100462 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06ed286144941269b9a0c0c60bff0ce" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:28.098343 11249 leader_election.cc:302] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1
I20260926 07:35:28.103281 11901 raft_consensus.cc:2781] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:28.103672 11901 raft_consensus.cc:482] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:28.104056 11901 raft_consensus.cc:3037] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:28.105305 11546 leader_election.cc:302] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:28.106266 11445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f581510fe624127a4d27d302f5be614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.108657 11435 tablet_service.cc:1467] Processing CreateTablet for tablet 5cd482fb159d418291cb02bc9138b076 (DEFAULT_TABLE table=default.Table_54b98a70017749499ab6c312bd7e6255 [id=3895daba9d194983a7018f02aa7b02a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.109773 11435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cd482fb159d418291cb02bc9138b076. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.111095 11286 tablet_service.cc:1467] Processing CreateTablet for tablet 5cd482fb159d418291cb02bc9138b076 (DEFAULT_TABLE table=default.Table_54b98a70017749499ab6c312bd7e6255 [id=3895daba9d194983a7018f02aa7b02a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.112073 11286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cd482fb159d418291cb02bc9138b076. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.112504 11444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd4f14abfea4799b75c48531412fc20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.112543 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 61) took 2506 ms. Trace:
I20260926 07:35:28.112895 11752 rpcz_store.cc:276] 0926 07:35:25.605842 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:25.605974 (+ 132us) service_pool.cc:227] Handling call
0926 07:35:27.369234 (+1763260us) ts_tablet_manager.cc:1145] Deleting tablet afa901cdc6a84220bf36ee669a955b72
0926 07:35:27.369297 (+ 63us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:27.859175 (+489878us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:28.060714 (+201539us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:28.112471 (+ 51757us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1762723}
I20260926 07:35:28.114225 11752 tablet_replica.cc:330] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:28.114848 11752 raft_consensus.cc:2229] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:28.115391 11752 pending_rounds.cc:62] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1: Trying to abort 1 pending ops.
I20260926 07:35:28.115597 11752 pending_rounds.cc:68] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333511689071390720 op_type: NO_OP noop_request { }
I20260926 07:35:28.116760 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 7000bcd55a984d64ba9d7d50fe63822c (table default.Table_4bc696441dce4bc687a0344b93eccb9e [id=12a935394c1f49158f105705ce038532]) successfully deleted
I20260926 07:35:28.120250 11315 raft_consensus.cc:2454] T e06ed286144941269b9a0c0c60bff0ce P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 1.
I20260926 07:35:28.120592 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet afa901cdc6a84220bf36ee669a955b72 (table default.Table_b5e830fcbaa44c209d50a84038d3b68f [id=be6bd29e04f342cbb7993b18e13d9332]) successfully deleted
I20260926 07:35:28.131703 11288 tablet_service.cc:1467] Processing CreateTablet for tablet cef2e44be4084fc78e61cf822659b4ea (DEFAULT_TABLE table=default.Table_ba602ce39d464e51bd0faa02b98f4363 [id=8a1a090674254074bc4db63f2c624eb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.131824 11439 tablet_service.cc:1467] Processing CreateTablet for tablet cef2e44be4084fc78e61cf822659b4ea (DEFAULT_TABLE table=default.Table_ba602ce39d464e51bd0faa02b98f4363 [id=8a1a090674254074bc4db63f2c624eb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.132756 11288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cef2e44be4084fc78e61cf822659b4ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.132951 11439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cef2e44be4084fc78e61cf822659b4ea. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:28.138399 11545 leader_election.cc:341] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:28.138439 11581 tablet_service.cc:1467] Processing CreateTablet for tablet 5cd482fb159d418291cb02bc9138b076 (DEFAULT_TABLE table=default.Table_54b98a70017749499ab6c312bd7e6255 [id=3895daba9d194983a7018f02aa7b02a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.140107 11581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cd482fb159d418291cb02bc9138b076. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.140775 11546 leader_election.cc:302] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1
I20260926 07:35:28.139410 11907 raft_consensus.cc:482] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:28.146662 11907 raft_consensus.cc:504] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:28.138228 11904 raft_consensus.cc:504] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:28.148916 11907 leader_election.cc:288] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:28.149358 11907 raft_consensus.cc:2781] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:28.149633 11907 raft_consensus.cc:482] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:28.149920 11907 raft_consensus.cc:3037] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:28.150069 11904 leader_election.cc:288] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:28.125249 11753 ts_tablet_manager.cc:1905] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 07:35:28.152546 11105 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet c81a59361a6e42b7962b3e027c4a087f (table default.Table_1b1901dc5bbb4288be18304dd4a9b113 [id=707c7ee5ced14cee8b8005cce80dbbb5], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet c81a59361a6e42b7962b3e027c4a087f consensus state has no leader
I20260926 07:35:28.115967 11752 raft_consensus.cc:2866] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 07:35:28.156415 11752 raft_consensus.cc:2258] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:28.158910 11752 ts_tablet_manager.cc:1905] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:28.155305 11285 tablet_service.cc:1467] Processing CreateTablet for tablet 248ab84a1f284c2bbddf62f00b485e13 (DEFAULT_TABLE table=default.Table_ba08930510174e65840452201a15f719 [id=47d4200172794cc78a6f386fee4b15b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.094700 11116 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet a0bdf6a4656e4217b0a7013fddc4cca8 (table default.Table_c7fb27daeba04d90bc0cf0c7ac07fe56 [id=3ce63f9fb5ad4b82ba2fbbf2e3678271]). Expected version 1 got 0
I20260926 07:35:28.160821 11285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 248ab84a1f284c2bbddf62f00b485e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.167269 11119 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet 4826318c83524f41840e770f33bd3696 (table default.Table_bbac1fe8bf474590ad44d694788a6c3a [id=55a6e4b7598043d8bf49512f2542c589]). Expected version 1 got 0
I20260926 07:35:28.168035 11901 raft_consensus.cc:504] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:28.171670 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75d0b86635944407b83f5ddfa29fcc4d" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
W20260926 07:35:28.172928 11247 leader_election.cc:341] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:28.173404 11901 leader_election.cc:288] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:28.175606 11446 tablet_service.cc:1467] Processing CreateTablet for tablet 248ab84a1f284c2bbddf62f00b485e13 (DEFAULT_TABLE table=default.Table_ba08930510174e65840452201a15f719 [id=47d4200172794cc78a6f386fee4b15b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.184883 11112 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: default.Table_d0380e9d3f26439bbc91cf476aeae69f [id=19875953f2b6484f934503404af57593] alter complete (version 2)
W20260926 07:35:28.187417 11112 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.1:34359 (request call id 36) took 1171 ms (client timeout 1000 ms). Trace:
W20260926 07:35:28.187738 11112 rpcz_store.cc:269] 0926 07:35:27.029966 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:27.239291 (+209325us) service_pool.cc:227] Handling call
0926 07:35:28.187357 (+948066us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:27.732405 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:27.733160 (+ 755us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:27.733185 (+ 25us) write_op.cc:435] Acquired schema lock
0926 07:35:27.733201 (+ 16us) tablet.cc:660] Decoding operations
0926 07:35:27.733509 (+ 308us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:27.733555 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:27.733582 (+ 27us) tablet.cc:683] Acquiring locks for 1 operations
0926 07:35:27.733738 (+ 156us) tablet.cc:699] Row locks acquired
0926 07:35:27.733754 (+ 16us) write_op.cc:260] PREPARE: finished
0926 07:35:27.733888 (+ 134us) write_op.cc:270] Start()
0926 07:35:27.733986 (+ 98us) write_op.cc:276] Timestamp: P: 1790408127733832 usec, L: 0
0926 07:35:27.734005 (+ 19us) op_driver.cc:331] REPLICATION: starting
0926 07:35:27.826174 (+ 92169us) log.cc:842] Serialized 356 byte log entry
0926 07:35:27.828099 (+ 1925us) spinlock_profiling.cc:243] Waited 91.6 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032b471 00007fb8103a2198 00007fb810803ade 00007fb8108032b8 00007fb810807ba1 00007fb810807b59 00007fb810807ae9 00007fb810807ab1 00007fb810806dad 0000000000580784 00000000005805b9
0926 07:35:27.844250 (+ 16151us) op_driver.cc:444] REPLICATION: finished
0926 07:35:27.844616 (+ 366us) write_op.cc:301] APPLY: starting
0926 07:35:27.844672 (+ 56us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:27.844763 (+ 91us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:27.844787 (+ 24us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:27.844943 (+ 156us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:27.845085 (+ 142us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:27.845131 (+ 46us) write_op.cc:312] APPLY: finished
0926 07:35:27.845305 (+ 174us) log.cc:842] Serialized 23 byte log entry
0926 07:35:27.845472 (+ 167us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:27.845533 (+ 61us) write_op.cc:454] Released schema lock
0926 07:35:27.845564 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:27.845734 (+ 170us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:27.845761 (+ 27us) write_op.cc:454] Released schema lock
0926 07:35:27.901756 (+ 55995us) spinlock_profiling.cc:243] Waited 62.3 ms on lock 0x7b58000b03b0. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb810320834 00007fb810325771 00007fb8103256f9 00007fb810325689 00007fb810325651 00007fb81032481d 0000000000580784 00000000005805b9 00007fb8094516b6 00007fb809454b11
Metrics: {"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":225,"apply.run_cpu_time_us":1288,"apply.run_wall_time_us":1288,"num_ops":1,"prepare.queue_time_us":463154,"prepare.run_cpu_time_us":3617,"prepare.run_wall_time_us":174609,"raft.queue_time_us":9974,"raft.run_cpu_time_us":2744,"raft.run_wall_time_us":73328,"replication_time_us":110166,"spinlock_wait_cycles":354006016}]]}
I20260926 07:35:28.176643 11446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 248ab84a1f284c2bbddf62f00b485e13. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:28.197652 11111 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.3:53335 (request call id 26) took 819 ms (client timeout 1000 ms). Trace:
W20260926 07:35:28.197992 11111 rpcz_store.cc:269] 0926 07:35:27.377787 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:27.443592 (+ 65805us) service_pool.cc:227] Handling call
0926 07:35:27.493534 (+ 49942us) spinlock_profiling.cc:243] Waited 37.6 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812374c3e 00007fb80feabfb4 00007fb80feabf42 00007fb80feabe71 00007fb80feabdec 00007fb80feab0a2 00007fb80e15594c 00007fb80e154d86
0926 07:35:28.197598 (+704064us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":6,"spinlock_wait_cycles":86416768}
I20260926 07:35:28.200204 11118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 1620b507f07c446c8c1811895a097409
I20260926 07:35:28.200337 11437 tablet_service.cc:1467] Processing CreateTablet for tablet 0f20206ce6fd4fd395f1b2b0154245db (DEFAULT_TABLE table=default.Table_a97a2f44056d4080b8eb45cb95106a2f [id=d0a7c7eb78ec434586bd93c0f8a313d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.201457 11437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f20206ce6fd4fd395f1b2b0154245db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.201472 11118 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:44836 (request call id 151) took 1858 ms. Trace:
W20260926 07:35:28.206619 11117 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.2:33317 (request call id 34) took 2361 ms (client timeout 1000 ms). Trace:
W20260926 07:35:28.206921 11117 rpcz_store.cc:269] 0926 07:35:25.845546 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:25.846484 (+ 938us) service_pool.cc:227] Handling call
0926 07:35:25.928741 (+ 82257us) spinlock_profiling.cc:243] Waited 80.4 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812374c3e 00007fb80feabfb4 00007fb80feabf42 00007fb80feabe71 00007fb80feabdec 00007fb80feab0a2 00007fb80e15594c 00007fb80e154d86
0926 07:35:28.206559 (+2277818us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":12255,"reported_tablets":3,"spinlock_wait_cycles":184937728}
I20260926 07:35:28.213761 11289 tablet_service.cc:1514] Processing DeleteTablet for tablet 1620b507f07c446c8c1811895a097409 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:26 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:28.201743 11118 rpcz_store.cc:276] 0926 07:35:26.342638 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.343495 (+ 857us) service_pool.cc:227] Handling call
0926 07:35:26.344892 (+ 1397us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0926 07:35:26.345411 (+ 519us) catalog_manager.cc:2752] Modifying in-memory table state
0926 07:35:26.345531 (+ 120us) catalog_manager.cc:2761] Locking tablets
0926 07:35:26.398103 (+ 52572us) catalog_manager.cc:2775] Removing table and tablets from system table
0926 07:35:27.438504 (+1040401us) catalog_manager.cc:2793] Removing table from by-name map
0926 07:35:27.438714 (+ 210us) catalog_manager.cc:2809] Committing in-memory state
0926 07:35:28.200095 (+761381us) catalog_manager.cc:2813] Aborting table tasks
0926 07:35:28.201404 (+ 1309us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:26.430107 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:26.430462 (+ 355us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:26.430494 (+ 32us) write_op.cc:435] Acquired schema lock
0926 07:35:26.430506 (+ 12us) tablet.cc:660] Decoding operations
0926 07:35:26.430779 (+ 273us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:26.430810 (+ 31us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:26.430830 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0926 07:35:26.431091 (+ 261us) tablet.cc:699] Row locks acquired
0926 07:35:26.431104 (+ 13us) write_op.cc:260] PREPARE: finished
0926 07:35:26.431198 (+ 94us) write_op.cc:270] Start()
0926 07:35:26.431269 (+ 71us) write_op.cc:276] Timestamp: P: 1790408126431138 usec, L: 0
0926 07:35:26.431283 (+ 14us) op_driver.cc:331] REPLICATION: starting
0926 07:35:26.431697 (+ 414us) log.cc:842] Serialized 759 byte log entry
0926 07:35:27.434297 (+1002600us) op_driver.cc:444] REPLICATION: finished
0926 07:35:27.435121 (+ 824us) write_op.cc:301] APPLY: starting
0926 07:35:27.435174 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:27.435270 (+ 96us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:27.435366 (+ 96us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:27.435569 (+ 203us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:27.435689 (+ 120us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0926 07:35:27.435730 (+ 41us) write_op.cc:312] APPLY: finished
0926 07:35:27.435884 (+ 154us) log.cc:842] Serialized 29 byte log entry
0926 07:35:27.436023 (+ 139us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:27.436085 (+ 62us) write_op.cc:454] Released schema lock
0926 07:35:27.436129 (+ 44us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:27.436714 (+ 585us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:27.436737 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":685,"apply.run_cpu_time_us":1367,"apply.run_wall_time_us":1784,"num_ops":2,"prepare.queue_time_us":30408,"prepare.run_cpu_time_us":1889,"prepare.run_wall_time_us":34714,"raft.queue_time_us":32527,"raft.run_cpu_time_us":21,"raft.run_wall_time_us":24,"replication_time_us":1002940}]]}
W20260926 07:35:28.215566 11113 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.2:33317 (request call id 35) took 1166 ms (client timeout 1000 ms). Trace:
W20260926 07:35:28.215885 11113 rpcz_store.cc:269] 0926 07:35:27.049111 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:27.241095 (+191984us) service_pool.cc:227] Handling call
0926 07:35:28.215505 (+974410us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":6}
I20260926 07:35:28.217413 11440 tablet_service.cc:1514] Processing DeleteTablet for tablet 1620b507f07c446c8c1811895a097409 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:26 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:28.217478 11842 raft_consensus.cc:482] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:28.218401 11285 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a255ef54f7e410ca8a779e5d7a118ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:23 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:28.219516 11842 raft_consensus.cc:504] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:28.221312 11842 leader_election.cc:288] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:28.222513 11440 tablet_service.cc:1514] Processing DeleteTablet for tablet 14e5da3711fe46c4bb46a7a60f41a581 with delete_type TABLET_DATA_DELETED (replaced by tablet 35f8ea999baa4d38a1433975ff6ccbb5) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:28.222815 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a255ef54f7e410ca8a779e5d7a118ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:23 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:28.223860 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0bdf6a4656e4217b0a7013fddc4cca8" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:28.224647 11315 raft_consensus.cc:2379] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 22f3902d01584936b20437606cb86dd1 in current term 1: Already voted for candidate 6070e9e3d1234a9d83e49431416afade in this term.
I20260926 07:35:28.229377 11756 tablet_bootstrap.cc:492] T 509f607b089c44bc9c3f80ed59a68c95 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:28.229838 11756 ts_tablet_manager.cc:1397] T 509f607b089c44bc9c3f80ed59a68c95 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.252s user 0.014s sys 0.004s
I20260926 07:35:28.236922 11756 raft_consensus.cc:348] T 509f607b089c44bc9c3f80ed59a68c95 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:28.237600 11756 raft_consensus.cc:374] T 509f607b089c44bc9c3f80ed59a68c95 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:28.237857 11756 raft_consensus.cc:729] T 509f607b089c44bc9c3f80ed59a68c95 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:28.239349 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_f6f34cb43e5c462aa22636e6daaf274e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:28.247532 11436 tablet_service.cc:1467] Processing CreateTablet for tablet 52d064b684d0491e91f176d24eddabe1 (DEFAULT_TABLE table=default.Table_8304288ba9e54e32a49193d760c6e068 [id=d19e4ba11fa14fa18c7ae93bf40f7c89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.243881 11756 consensus_queue.cc:260] T 509f607b089c44bc9c3f80ed59a68c95 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:28.248780 11436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52d064b684d0491e91f176d24eddabe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.245311 11114 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 4239f5d48512403bb711ece401f0756b
I20260926 07:35:28.248361 11120 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 04be706330924282961054c5835cec73
I20260926 07:35:28.252373 11120 catalog_manager.cc:6565] ReplaceTablet: tablet 04be706330924282961054c5835cec73 deleted and replaced by tablet 21919d22b1d24546911d36384b01d9cc
I20260926 07:35:28.252673 11120 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:44836 (request call id 160) took 1076 ms. Trace:
I20260926 07:35:28.255029 11285 tablet_service.cc:1467] Processing CreateTablet for tablet 0f20206ce6fd4fd395f1b2b0154245db (DEFAULT_TABLE table=default.Table_a97a2f44056d4080b8eb45cb95106a2f [id=d0a7c7eb78ec434586bd93c0f8a313d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.252944 11120 rpcz_store.cc:276] 0926 07:35:27.176106 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:27.469647 (+293541us) service_pool.cc:227] Handling call
0926 07:35:27.479688 (+ 10041us) spinlock_profiling.cc:243] Waited 9.14 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812378b81 00007fb80fed3d37 00007fb80fed3cc2 00007fb80fed3bf1 00007fb80fed3b6c 00007fb80fed2e22 00007fb80e15594c 00007fb80e154d86
0926 07:35:28.252627 (+772939us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:28.105350 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:28.105767 (+ 417us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:28.105796 (+ 29us) write_op.cc:435] Acquired schema lock
0926 07:35:28.105817 (+ 21us) tablet.cc:660] Decoding operations
0926 07:35:28.106192 (+ 375us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:28.106241 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:28.106270 (+ 29us) tablet.cc:683] Acquiring locks for 2 operations
0926 07:35:28.106500 (+ 230us) tablet.cc:699] Row locks acquired
0926 07:35:28.106518 (+ 18us) write_op.cc:260] PREPARE: finished
0926 07:35:28.106646 (+ 128us) write_op.cc:270] Start()
0926 07:35:28.106728 (+ 82us) write_op.cc:276] Timestamp: P: 1790408128106565 usec, L: 0
0926 07:35:28.106748 (+ 20us) op_driver.cc:331] REPLICATION: starting
0926 07:35:28.107243 (+ 495us) log.cc:842] Serialized 625 byte log entry
0926 07:35:28.240238 (+132995us) op_driver.cc:444] REPLICATION: finished
0926 07:35:28.242101 (+ 1863us) write_op.cc:301] APPLY: starting
0926 07:35:28.242156 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:28.242257 (+ 101us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:28.242345 (+ 88us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:28.242656 (+ 311us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:28.242895 (+ 239us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:28.242954 (+ 59us) write_op.cc:312] APPLY: finished
0926 07:35:28.243133 (+ 179us) log.cc:842] Serialized 29 byte log entry
0926 07:35:28.243266 (+ 133us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:28.243328 (+ 62us) write_op.cc:454] Released schema lock
0926 07:35:28.243357 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:28.243518 (+ 161us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:28.243541 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":21032832,"child_traces":[["op",{"apply.queue_time_us":1699,"apply.run_cpu_time_us":1609,"apply.run_wall_time_us":1609,"num_ops":2,"prepare.queue_time_us":78571,"prepare.run_cpu_time_us":2244,"prepare.run_wall_time_us":44803,"raft.queue_time_us":67763,"raft.run_cpu_time_us":639,"raft.run_wall_time_us":640,"replication_time_us":133410}]]}
I20260926 07:35:28.256042 11285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f20206ce6fd4fd395f1b2b0154245db. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:28.227231 11546 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.1:41829, user_credentials={real_user=slave}} blocked reactor thread for 87444.7us
W20260926 07:35:28.183413 11545 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.171.2:42379, user_credentials={real_user=slave}} blocked reactor thread for 44312.3us
W20260926 07:35:28.259620 11112 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f6f34cb43e5c462aa22636e6daaf274e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:28.260502 11295 tablet_service.cc:1514] Processing DeleteTablet for tablet acc5568dee9b473f8f5f1f8bc3b5504e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:23 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:28.265738 11445 tablet_service.cc:1514] Processing DeleteTablet for tablet 04be706330924282961054c5835cec73 with delete_type TABLET_DATA_DELETED (replaced by tablet 21919d22b1d24546911d36384b01d9cc) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:28.266963 11111 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_13ddd864a4c2474abb61d8c3b975b8d7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:28.269100 11111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_13ddd864a4c2474abb61d8c3b975b8d7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:28.270853 11753 ts_tablet_manager.cc:1918] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:28.271255 11753 log.cc:1196] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/14e5da3711fe46c4bb46a7a60f41a581
I20260926 07:35:28.272050 11753 ts_tablet_manager.cc:1939] T 14e5da3711fe46c4bb46a7a60f41a581 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:28.274153 11295 tablet_service.cc:1514] Processing DeleteTablet for tablet 4239f5d48512403bb711ece401f0756b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:28 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:28.276623 11289 tablet_service.cc:1514] Processing DeleteTablet for tablet 04be706330924282961054c5835cec73 with delete_type TABLET_DATA_DELETED (replaced by tablet 21919d22b1d24546911d36384b01d9cc) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:28.290277 11117 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_bbac1fe8bf474590ad44d694788a6c3a" } new_table_name: "default.Table_0bc40fffac774a26bf552c999331f2d4" modify_external_catalogs: true
I20260926 07:35:28.292953 11115 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_a1a81954a9d34a709916de820cfb9952"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:28.293583 11114 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_c961864a78e341a9bf757eaebadc4e84" } new_table_name: "default.Table_cd53ff22f9444f688e341de983e3e145" modify_external_catalogs: true
I20260926 07:35:28.295456 11753 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 64) took 2662 ms. Trace:
W20260926 07:35:28.299355 11115 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a1a81954a9d34a709916de820cfb9952 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:28.312984 11116 master_service.cc:946] ReplaceTablet: received request to replace tablet acc5568dee9b473f8f5f1f8bc3b5504e from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:28.312956 11288 tablet_service.cc:1467] Processing CreateTablet for tablet 52d064b684d0491e91f176d24eddabe1 (DEFAULT_TABLE table=default.Table_8304288ba9e54e32a49193d760c6e068 [id=d19e4ba11fa14fa18c7ae93bf40f7c89]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 07:35:28.295869 11104 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.171.62:37629, user_credentials={real_user=slave}} blocked reactor thread for 69342.2us
I20260926 07:35:28.314316 11288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52d064b684d0491e91f176d24eddabe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.295758 11753 rpcz_store.cc:276] 0926 07:35:25.633202 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:25.633359 (+ 157us) service_pool.cc:227] Handling call
0926 07:35:28.070672 (+2437313us) ts_tablet_manager.cc:1145] Deleting tablet 14e5da3711fe46c4bb46a7a60f41a581
0926 07:35:28.070706 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:28.188919 (+118213us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:28.270620 (+ 81701us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:28.295398 (+ 24778us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2354315}
I20260926 07:35:28.317061 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet 14e5da3711fe46c4bb46a7a60f41a581 (table default.Table_ff479f29ec0143d6a74b38e25389f1bc [id=2238dbbde77c45cd8689ba10e6dbf472]) successfully deleted
I20260926 07:35:28.317214 11753 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 65) took 2682 ms. Trace:
I20260926 07:35:28.317494 11753 rpcz_store.cc:276] 0926 07:35:25.634535 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:25.634684 (+ 149us) service_pool.cc:227] Handling call
0926 07:35:28.316885 (+2682201us) ts_tablet_manager.cc:1145] Deleting tablet 509f607b089c44bc9c3f80ed59a68c95
0926 07:35:28.316922 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:28.317169 (+ 247us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2600347}
I20260926 07:35:28.318514 11753 tablet_replica.cc:330] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:28.319182 11753 raft_consensus.cc:2229] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Raft consensus shutting down.
W20260926 07:35:28.322142 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 509f607b089c44bc9c3f80ed59a68c95 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 509f607b089c44bc9c3f80ed59a68c95 already in progress: creating tablet
I20260926 07:35:28.323239 11746 tablet_bootstrap.cc:492] T 4826318c83524f41840e770f33bd3696 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:28.323567 11756 ts_tablet_manager.cc:1428] T 509f607b089c44bc9c3f80ed59a68c95 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.093s user 0.007s sys 0.000s
I20260926 07:35:28.323629 11746 ts_tablet_manager.cc:1397] T 4826318c83524f41840e770f33bd3696 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.724s user 0.011s sys 0.011s
I20260926 07:35:28.270143 11444 tablet_service.cc:1514] Processing DeleteTablet for tablet 4239f5d48512403bb711ece401f0756b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:28 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:28.326023 11746 raft_consensus.cc:348] T 4826318c83524f41840e770f33bd3696 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:28.326664 11746 raft_consensus.cc:374] T 4826318c83524f41840e770f33bd3696 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:28.326907 11746 raft_consensus.cc:729] T 4826318c83524f41840e770f33bd3696 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:28.327510 11746 consensus_queue.cc:260] T 4826318c83524f41840e770f33bd3696 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:28.333865 11292 tablet_service.cc:1514] Processing DeleteTablet for tablet e06ed286144941269b9a0c0c60bff0ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:24 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:28.332662 11756 tablet_bootstrap.cc:492] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:28.336951 11754 ts_tablet_manager.cc:1428] T a55e54d492214039b0294f954b5ef390 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.350s user 0.006s sys 0.020s
I20260926 07:35:28.337853 11754 tablet_bootstrap.cc:492] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
W20260926 07:35:28.342154 11117 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4826318c83524f41840e770f33bd3696 consensus state has no leader
I20260926 07:35:28.319983 11753 raft_consensus.cc:2258] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:28.353238 11746 ts_tablet_manager.cc:1428] T 4826318c83524f41840e770f33bd3696 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.029s user 0.005s sys 0.001s
I20260926 07:35:28.354135 11746 tablet_bootstrap.cc:492] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:28.357474 11112 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 4826318c83524f41840e770f33bd3696 (table default.Table_0bc40fffac774a26bf552c999331f2d4 [id=55a6e4b7598043d8bf49512f2542c589]). Expected version 2 got 0
W20260926 07:35:28.374908 11114 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 640d21586bad4070957cff61bd174fd2 consensus state has no leader
I20260926 07:35:28.377228 11746 tablet_bootstrap.cc:654] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:28.380406 11112 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 4826318c83524f41840e770f33bd3696 (table default.Table_0bc40fffac774a26bf552c999331f2d4 [id=55a6e4b7598043d8bf49512f2542c589]). Expected version 2 got 0
I20260926 07:35:28.381857 11969 raft_consensus.cc:2781] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:28.395294 11753 ts_tablet_manager.cc:1905] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:28.417572 11112 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_a1a81954a9d34a709916de820cfb9952" } modify_external_catalogs: true
I20260926 07:35:28.422298 11292 tablet_service.cc:1514] Processing DeleteTablet for tablet 14e5da3711fe46c4bb46a7a60f41a581 with delete_type TABLET_DATA_DELETED (replaced by tablet 35f8ea999baa4d38a1433975ff6ccbb5) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:28.420892 11112 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_a1a81954a9d34a709916de820cfb9952" } modify_external_catalogs: true
I20260926 07:35:28.446450 11285 tablet_service.cc:1514] Processing DeleteTablet for tablet aa02661e7531402687f03e86dbc19418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:23 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:28.447564 11285 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e1cea4523a64428aac03ca0bdf708c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:24 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:28.455210 11115 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_1b1901dc5bbb4288be18304dd4a9b113" } modify_external_catalogs: true
I20260926 07:35:28.455936 11115 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_1b1901dc5bbb4288be18304dd4a9b113" } modify_external_catalogs: true
I20260926 07:35:28.459873 11746 tablet_bootstrap.cc:492] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:28.467119 11114 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_5c53cf2e6f714b1db5df09a40f51c11a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:28.468559 11114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5c53cf2e6f714b1db5df09a40f51c11a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:28.460439 11746 ts_tablet_manager.cc:1397] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.106s user 0.009s sys 0.007s
I20260926 07:35:28.474746 11746 raft_consensus.cc:348] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:28.475461 11746 raft_consensus.cc:374] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:28.491854 11746 raft_consensus.cc:729] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:28.502082 11753 ts_tablet_manager.cc:1918] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:28.509290 11753 log.cc:1196] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/067ae4ea43f54e88874d93b4393dae27
I20260926 07:35:28.510124 11753 ts_tablet_manager.cc:1939] T 067ae4ea43f54e88874d93b4393dae27 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:28.517406 11753 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 66) took 2881 ms. Trace:
I20260926 07:35:28.517706 11753 rpcz_store.cc:276] 0926 07:35:25.635641 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:25.635742 (+ 101us) service_pool.cc:227] Handling call
0926 07:35:28.318387 (+2682645us) ts_tablet_manager.cc:1145] Deleting tablet 067ae4ea43f54e88874d93b4393dae27
0926 07:35:28.318423 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:28.440270 (+121847us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:28.501872 (+ 61602us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:28.517358 (+ 15486us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2601721}
I20260926 07:35:28.518869 11753 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 70) took 2520 ms. Trace:
I20260926 07:35:28.519146 11753 rpcz_store.cc:276] 0926 07:35:25.998880 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:25.998994 (+ 114us) service_pool.cc:227] Handling call
0926 07:35:28.518559 (+2519565us) ts_tablet_manager.cc:1145] Deleting tablet 55ad1d96e2314d1eb0a717a3e9ff31bc
0926 07:35:28.518595 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:28.518824 (+ 229us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2518932}
I20260926 07:35:28.513703 11756 tablet_bootstrap.cc:654] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:28.520192 11753 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 69) took 2521 ms. Trace:
I20260926 07:35:28.492977 11746 consensus_queue.cc:260] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:28.521152 11753 rpcz_store.cc:276] 0926 07:35:25.998606 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.001593 (+ 2987us) service_pool.cc:227] Handling call
0926 07:35:28.519857 (+2518264us) ts_tablet_manager.cc:1145] Deleting tablet a8b629a4aba045b697e33bbebecb05b7
0926 07:35:28.519896 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:28.520144 (+ 248us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2517265}
I20260926 07:35:28.521849 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet 067ae4ea43f54e88874d93b4393dae27 (table default.Table_bc4fc15cb6014a1e86ba2886b2950bae [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b]) successfully deleted
W20260926 07:35:28.522734 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 55ad1d96e2314d1eb0a717a3e9ff31bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55ad1d96e2314d1eb0a717a3e9ff31bc already in progress: creating tablet
I20260926 07:35:28.522933 11753 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 75) took 2104 ms. Trace:
I20260926 07:35:28.523203 11753 rpcz_store.cc:276] 0926 07:35:26.418461 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.420333 (+ 1872us) service_pool.cc:227] Handling call
0926 07:35:28.522648 (+2102315us) ts_tablet_manager.cc:1145] Deleting tablet d6fc6178f1e54c3984a20873864233cd
0926 07:35:28.522682 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:28.522889 (+ 207us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2101745}
I20260926 07:35:28.523998 11753 tablet_replica.cc:330] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
W20260926 07:35:28.524230 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet a8b629a4aba045b697e33bbebecb05b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a8b629a4aba045b697e33bbebecb05b7 already in progress: creating tablet
I20260926 07:35:28.524716 11753 raft_consensus.cc:2229] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Raft consensus shutting down.
W20260926 07:35:28.524840 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet d6fc6178f1e54c3984a20873864233cd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d6fc6178f1e54c3984a20873864233cd already in progress: creating tablet
I20260926 07:35:28.525386 11753 raft_consensus.cc:2258] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:28.527107 11753 ts_tablet_manager.cc:1905] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:28.548094 11119 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_0bc40fffac774a26bf552c999331f2d4" } new_table_name: "default.Table_0cf0eeab44174032830f060f976b982e" modify_external_catalogs: true
I20260926 07:35:28.548800 11116 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_e9004884cb974d669eae904cc826d9fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:28.550390 11116 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e9004884cb974d669eae904cc826d9fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:28.555954 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e1cea4523a64428aac03ca0bdf708c7" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:28.556569 11315 raft_consensus.cc:2454] T 1e1cea4523a64428aac03ca0bdf708c7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 0.
I20260926 07:35:28.579362 11117 master_service.cc:946] ReplaceTablet: received request to replace tablet d6fc6178f1e54c3984a20873864233cd from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:28.584893 11753 ts_tablet_manager.cc:1918] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:28.590145 11753 log.cc:1196] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/f31a069dc888487ebbbb1cb91e886ad0
I20260926 07:35:28.591785 11753 ts_tablet_manager.cc:1939] T f31a069dc888487ebbbb1cb91e886ad0 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:28.594367 11753 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 76) took 2175 ms. Trace:
I20260926 07:35:28.596590 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4826318c83524f41840e770f33bd3696" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:28.597291 11315 raft_consensus.cc:3037] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:28.597272 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0bdf6a4656e4217b0a7013fddc4cca8" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
W20260926 07:35:28.599203 11589 consensus_meta.cc:287] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade: Time spent flushing consensus metadata: real 0.501s user 0.011s sys 0.001s
I20260926 07:35:28.604138 11315 raft_consensus.cc:2454] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 1.
I20260926 07:35:28.604928 11753 rpcz_store.cc:276] 0926 07:35:26.419108 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.424213 (+ 5105us) service_pool.cc:227] Handling call
0926 07:35:28.523877 (+2099664us) ts_tablet_manager.cc:1145] Deleting tablet f31a069dc888487ebbbb1cb91e886ad0
0926 07:35:28.523909 (+ 32us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:28.560965 (+ 37056us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:28.584674 (+ 23709us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:28.594314 (+ 9640us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2099001}
I20260926 07:35:28.607879 11753 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 85) took 1117 ms. Trace:
I20260926 07:35:28.607897 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet f31a069dc888487ebbbb1cb91e886ad0 (table default.Table_ba08930510174e65840452201a15f719 [id=47d4200172794cc78a6f386fee4b15b4]) successfully deleted
I20260926 07:35:28.609922 11753 rpcz_store.cc:276] 0926 07:35:27.490073 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:27.491086 (+ 1013us) service_pool.cc:227] Handling call
0926 07:35:28.607569 (+1116483us) ts_tablet_manager.cc:1145] Deleting tablet c36c759963cb4915afdca7d12109cc6e
0926 07:35:28.607607 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:28.607832 (+ 225us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1055235}
I20260926 07:35:28.610869 11753 tablet_replica.cc:330] T 1620b507f07c446c8c1811895a097409 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
W20260926 07:35:28.611111 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet c36c759963cb4915afdca7d12109cc6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c36c759963cb4915afdca7d12109cc6e already in progress: creating tablet
I20260926 07:35:28.611519 11753 raft_consensus.cc:2229] T 1620b507f07c446c8c1811895a097409 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:28.611928 11753 raft_consensus.cc:2258] T 1620b507f07c446c8c1811895a097409 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:28.516919 11752 ts_tablet_manager.cc:1918] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:28.543254 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e1cea4523a64428aac03ca0bdf708c7" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:28.593477 11842 raft_consensus.cc:482] T 4826318c83524f41840e770f33bd3696 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:28.595037 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4826318c83524f41840e770f33bd3696" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:28.616416 11842 raft_consensus.cc:504] T 4826318c83524f41840e770f33bd3696 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:28.618264 11753 ts_tablet_manager.cc:1905] T 1620b507f07c446c8c1811895a097409 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:28.618330 11842 leader_election.cc:288] T 4826318c83524f41840e770f33bd3696 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:28.619601 11754 tablet_bootstrap.cc:700] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade: Time spent opening tablet: real 0.232s user 0.006s sys 0.007s
I20260926 07:35:28.619773 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4826318c83524f41840e770f33bd3696" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:28.620496 11463 raft_consensus.cc:3037] T 4826318c83524f41840e770f33bd3696 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:28.620651 11754 tablet_bootstrap.cc:654] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:28.620730 11315 raft_consensus.cc:2379] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 22f3902d01584936b20437606cb86dd1 in current term 1: Already voted for candidate 6070e9e3d1234a9d83e49431416afade in this term.
I20260926 07:35:28.600327 11402 leader_election.cc:302] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 22f3902d01584936b20437606cb86dd1; no voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:28.632730 11983 raft_consensus.cc:3037] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:28.638491 11969 raft_consensus.cc:686] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Becoming Leader. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Running, Role: LEADER
I20260926 07:35:28.641924 11983 raft_consensus.cc:2726] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:28.641324 11969 consensus_queue.cc:237] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:28.649773 11904 raft_consensus.cc:482] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:28.650234 11904 raft_consensus.cc:504] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:28.652415 11970 raft_consensus.cc:482] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:28.652951 11970 raft_consensus.cc:504] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
W20260926 07:35:28.654266 11907 consensus_meta.cc:287] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade: Time spent flushing consensus metadata: real 0.504s user 0.012s sys 0.013s
I20260926 07:35:28.654980 11970 leader_election.cc:288] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:28.654749 11907 raft_consensus.cc:504] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:28.655891 11904 leader_election.cc:288] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:28.656368 11752 log.cc:1196] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/acc5568dee9b473f8f5f1f8bc3b5504e
I20260926 07:35:28.657665 11752 ts_tablet_manager.cc:1939] T acc5568dee9b473f8f5f1f8bc3b5504e P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:28.660956 11463 raft_consensus.cc:2454] T 4826318c83524f41840e770f33bd3696 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 1.
I20260926 07:35:28.664568 11589 tablet_service.cc:1467] Processing CreateTablet for tablet cef2e44be4084fc78e61cf822659b4ea (DEFAULT_TABLE table=default.Table_ba602ce39d464e51bd0faa02b98f4363 [id=8a1a090674254074bc4db63f2c624eb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.665913 11589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cef2e44be4084fc78e61cf822659b4ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.668361 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75d0b86635944407b83f5ddfa29fcc4d" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:28.668882 11609 raft_consensus.cc:3037] T 75d0b86635944407b83f5ddfa29fcc4d P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:28.677999 11746 ts_tablet_manager.cc:1428] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.205s user 0.005s sys 0.000s
I20260926 07:35:28.679656 11609 raft_consensus.cc:2454] T 75d0b86635944407b83f5ddfa29fcc4d P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 1.
I20260926 07:35:28.684882 11753 ts_tablet_manager.cc:1918] T 1620b507f07c446c8c1811895a097409 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:28.684756 11120 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_9444fa8f3e7e4c7e9a245b34db0fb9f1" } new_table_name: "default.Table_b90378c291a04ddda6fba25512c07fb4" modify_external_catalogs: true
I20260926 07:35:28.685209 11753 log.cc:1196] T 1620b507f07c446c8c1811895a097409 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/1620b507f07c446c8c1811895a097409
I20260926 07:35:28.685767 11753 ts_tablet_manager.cc:1939] T 1620b507f07c446c8c1811895a097409 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:28.685954 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4826318c83524f41840e770f33bd3696" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:28.686635 11608 raft_consensus.cc:2379] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 22f3902d01584936b20437606cb86dd1 in current term 1: Already voted for candidate 6070e9e3d1234a9d83e49431416afade in this term.
I20260926 07:35:28.688608 11118 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_4419f8321f5e4c0ebd5b75bb80a5d133"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:28.690122 11118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4419f8321f5e4c0ebd5b75bb80a5d133 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:28.693307 11753 tablet_replica.cc:330] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:28.693277 11907 leader_election.cc:288] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:28.694027 11753 raft_consensus.cc:2229] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:28.694509 11753 pending_rounds.cc:62] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e: Trying to abort 1 pending ops.
I20260926 07:35:28.694429 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 62) took 3087 ms. Trace:
I20260926 07:35:28.694702 11753 pending_rounds.cc:68] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333511689972666368 op_type: NO_OP noop_request { }
I20260926 07:35:28.695016 11753 raft_consensus.cc:2866] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 07:35:28.694720 11752 rpcz_store.cc:276] 0926 07:35:25.606840 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:25.606956 (+ 116us) service_pool.cc:227] Handling call
0926 07:35:28.114074 (+2507118us) ts_tablet_manager.cc:1145] Deleting tablet acc5568dee9b473f8f5f1f8bc3b5504e
0926 07:35:28.114127 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:28.245774 (+131647us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:28.516683 (+270909us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:28.694372 (+177689us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2506598}
I20260926 07:35:28.695300 11753 raft_consensus.cc:2258] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:28.695832 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet acc5568dee9b473f8f5f1f8bc3b5504e (table default.Table_23c2afadc1164dfab1f47d7c0b373e5f [id=6806510fcf2d4b17960903772e08d49b]) successfully deleted
I20260926 07:35:28.695824 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet 1620b507f07c446c8c1811895a097409 (table default.Table_0a660d82b80c40d8a4c41b9abdec566f [id=f23a7ed0439c41149de764ab244ef60b]) successfully deleted
I20260926 07:35:28.698132 11316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acc5568dee9b473f8f5f1f8bc3b5504e" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:28.698285 11249 leader_election.cc:302] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1
I20260926 07:35:28.698683 11316 raft_consensus.cc:2454] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 1.
I20260926 07:35:28.699218 11746 tablet_bootstrap.cc:492] T 75d0b86635944407b83f5ddfa29fcc4d P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:28.699267 11979 raft_consensus.cc:2781] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:28.699676 11979 raft_consensus.cc:686] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Becoming Leader. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Running, Role: LEADER
I20260926 07:35:28.700626 11979 consensus_queue.cc:237] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:28.702440 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_02974e73af5f4521a34bcb9d2c75a84a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:28.704078 11113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_02974e73af5f4521a34bcb9d2c75a84a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:28.709761 11436 tablet_service.cc:1514] Processing DeleteTablet for tablet e06ed286144941269b9a0c0c60bff0ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:24 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:28.713351 11753 ts_tablet_manager.cc:1905] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:28.715499 11754 tablet_bootstrap.cc:492] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:28.715915 11754 ts_tablet_manager.cc:1397] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.378s user 0.011s sys 0.012s
I20260926 07:35:28.718358 11588 tablet_service.cc:1467] Processing CreateTablet for tablet 0f20206ce6fd4fd395f1b2b0154245db (DEFAULT_TABLE table=default.Table_a97a2f44056d4080b8eb45cb95106a2f [id=d0a7c7eb78ec434586bd93c0f8a313d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.719516 11588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f20206ce6fd4fd395f1b2b0154245db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.729353 11583 tablet_service.cc:1467] Processing CreateTablet for tablet 248ab84a1f284c2bbddf62f00b485e13 (DEFAULT_TABLE table=default.Table_ba08930510174e65840452201a15f719 [id=47d4200172794cc78a6f386fee4b15b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.730438 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 248ab84a1f284c2bbddf62f00b485e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.732831 11117 master_service.cc:946] ReplaceTablet: received request to replace tablet 640d21586bad4070957cff61bd174fd2 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:28.734577 11977 raft_consensus.cc:482] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:28.735015 11977 raft_consensus.cc:504] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:28.736960 11977 leader_election.cc:288] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:28.744227 11756 tablet_bootstrap.cc:492] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:28.744652 11756 ts_tablet_manager.cc:1397] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.412s user 0.010s sys 0.000s
I20260926 07:35:28.746816 11756 raft_consensus.cc:348] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:28.747562 11756 raft_consensus.cc:374] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:28.745549 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a55e54d492214039b0294f954b5ef390" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
W20260926 07:35:28.749233 11247 leader_election.cc:341] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:28.751472 11436 tablet_service.cc:1467] Processing CreateTablet for tablet b8e926ee35854c28a8c698384526fe38 (DEFAULT_TABLE table=default.Table_38183654ed414223a1467d57fc2d7d29 [id=f9eb5434741f40528f4749a0096ead3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.753093 11546 leader_election.cc:302] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:28.753906 11546 leader_election.cc:302] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:28.754704 11963 raft_consensus.cc:2781] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:28.754815 11436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8e926ee35854c28a8c698384526fe38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.755015 11963 raft_consensus.cc:482] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:28.755290 11963 raft_consensus.cc:3037] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:28.757680 11288 tablet_service.cc:1467] Processing CreateTablet for tablet b8e926ee35854c28a8c698384526fe38 (DEFAULT_TABLE table=default.Table_38183654ed414223a1467d57fc2d7d29 [id=f9eb5434741f40528f4749a0096ead3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.758623 11288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8e926ee35854c28a8c698384526fe38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.763054 11402 leader_election.cc:302] T 4826318c83524f41840e770f33bd3696 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 22f3902d01584936b20437606cb86dd1; no voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:28.765210 11983 raft_consensus.cc:2726] T 4826318c83524f41840e770f33bd3696 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:28.765309 11285 tablet_service.cc:1514] Processing DeleteTablet for tablet 509f607b089c44bc9c3f80ed59a68c95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:28.766403 11292 tablet_service.cc:1467] Processing CreateTablet for tablet 17871fbd15064daba6e80a34a1b221d3 (DEFAULT_TABLE table=default.Table_8c8e3eb07a9d4b18a589cf8940bbe83f [id=b1dad62530414b119e059c427ccb2130]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.767642 11286 tablet_service.cc:1467] Processing CreateTablet for tablet f8f025d645eb478f95fa0aef17a86729 (DEFAULT_TABLE table=default.Table_00b32220835a4f1681581a6c9a426f17 [id=072e29bef31847c9beaa569621e5fe7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.768707 11292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17871fbd15064daba6e80a34a1b221d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.770159 11439 tablet_service.cc:1467] Processing CreateTablet for tablet 17871fbd15064daba6e80a34a1b221d3 (DEFAULT_TABLE table=default.Table_8c8e3eb07a9d4b18a589cf8940bbe83f [id=b1dad62530414b119e059c427ccb2130]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.771199 11439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17871fbd15064daba6e80a34a1b221d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.771134 11590 tablet_service.cc:1514] Processing DeleteTablet for tablet 067ae4ea43f54e88874d93b4393dae27 with delete_type TABLET_DATA_DELETED (replaced by tablet 0218d2a727f942e48d770878e6bf4a54) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:28.772212 11590 tablet_service.cc:1514] Processing DeleteTablet for tablet 14e5da3711fe46c4bb46a7a60f41a581 with delete_type TABLET_DATA_DELETED (replaced by tablet 35f8ea999baa4d38a1433975ff6ccbb5) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:28.772799 11446 tablet_service.cc:1467] Processing CreateTablet for tablet f8f025d645eb478f95fa0aef17a86729 (DEFAULT_TABLE table=default.Table_00b32220835a4f1681581a6c9a426f17 [id=072e29bef31847c9beaa569621e5fe7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.773085 11590 tablet_service.cc:1514] Processing DeleteTablet for tablet 1620b507f07c446c8c1811895a097409 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:26 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:28.773739 11446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8f025d645eb478f95fa0aef17a86729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.776952 11112 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 4f36a9f558694e9a85e406d0cd153296
I20260926 07:35:28.778901 11286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8f025d645eb478f95fa0aef17a86729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.787400 11907 raft_consensus.cc:2781] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:28.788108 11907 raft_consensus.cc:686] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Becoming Leader. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Running, Role: LEADER
W20260926 07:35:28.788344 11119 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4826318c83524f41840e770f33bd3696 consensus state has no leader
I20260926 07:35:28.788995 11907 consensus_queue.cc:237] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:28.791805 11590 tablet_service.cc:1514] Processing DeleteTablet for tablet 4239f5d48512403bb711ece401f0756b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:28 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:28.792718 11752 tablet_replica.cc:330] T 067ae4ea43f54e88874d93b4393dae27 P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:28.792743 11590 tablet_service.cc:1514] Processing DeleteTablet for tablet 04be706330924282961054c5835cec73 with delete_type TABLET_DATA_DELETED (replaced by tablet 21919d22b1d24546911d36384b01d9cc) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:28.793751 11752 raft_consensus.cc:2229] T 067ae4ea43f54e88874d93b4393dae27 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:28.794174 11752 raft_consensus.cc:2258] T 067ae4ea43f54e88874d93b4393dae27 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:28.794059 11590 tablet_service.cc:1467] Processing CreateTablet for tablet b8e926ee35854c28a8c698384526fe38 (DEFAULT_TABLE table=default.Table_38183654ed414223a1467d57fc2d7d29 [id=f9eb5434741f40528f4749a0096ead3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.795059 11590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8e926ee35854c28a8c698384526fe38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.798193 11437 tablet_service.cc:1467] Processing CreateTablet for tablet ed9f7faf55684e3e8512a033aa1443fe (DEFAULT_TABLE table=default.Table_8304ef6caf564378a58911a50bc8d9f7 [id=66cecf81391644ab84399e0f65fa4068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.799245 11437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed9f7faf55684e3e8512a033aa1443fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.747983 11756 raft_consensus.cc:729] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:28.805287 11581 tablet_service.cc:1467] Processing CreateTablet for tablet 52d064b684d0491e91f176d24eddabe1 (DEFAULT_TABLE table=default.Table_8304288ba9e54e32a49193d760c6e068 [id=d19e4ba11fa14fa18c7ae93bf40f7c89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.806325 11581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52d064b684d0491e91f176d24eddabe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.805823 11756 consensus_queue.cc:260] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:28.808624 11752 ts_tablet_manager.cc:1905] T 067ae4ea43f54e88874d93b4393dae27 P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:28.809406 11871 ts_tablet_manager.cc:1918] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:28.809960 11871 log.cc:1196] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/50d0e4cd192c445bb7c82d1a3c4a53d1
I20260926 07:35:28.810819 11871 ts_tablet_manager.cc:1939] T 50d0e4cd192c445bb7c82d1a3c4a53d1 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:28.812752 11285 tablet_service.cc:1467] Processing CreateTablet for tablet ed9f7faf55684e3e8512a033aa1443fe (DEFAULT_TABLE table=default.Table_8304ef6caf564378a58911a50bc8d9f7 [id=66cecf81391644ab84399e0f65fa4068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.813791 11285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed9f7faf55684e3e8512a033aa1443fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.816429 11435 tablet_service.cc:1467] Processing CreateTablet for tablet 21919d22b1d24546911d36384b01d9cc (DEFAULT_TABLE table=default.Table_4eb22a54d0df4f808fe85886a5809ab7 [id=cab439115f3c46d089824983fe26d0eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.817437 11435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21919d22b1d24546911d36384b01d9cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.825582 11290 tablet_service.cc:1467] Processing CreateTablet for tablet 21919d22b1d24546911d36384b01d9cc (DEFAULT_TABLE table=default.Table_4eb22a54d0df4f808fe85886a5809ab7 [id=cab439115f3c46d089824983fe26d0eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.826606 11290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21919d22b1d24546911d36384b01d9cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.834479 11756 ts_tablet_manager.cc:1428] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.090s user 0.005s sys 0.000s
I20260926 07:35:28.835310 11756 tablet_bootstrap.cc:492] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:28.835078 11444 tablet_service.cc:1467] Processing CreateTablet for tablet 6d524be8509645a89a75b4bac15f0889 (DEFAULT_TABLE table=default.Table_f6f34cb43e5c462aa22636e6daaf274e [id=9fae0d37b0f8491ab8626e1f523e29ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.836032 11444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d524be8509645a89a75b4bac15f0889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.839831 11289 tablet_service.cc:1467] Processing CreateTablet for tablet 6d524be8509645a89a75b4bac15f0889 (DEFAULT_TABLE table=default.Table_f6f34cb43e5c462aa22636e6daaf274e [id=9fae0d37b0f8491ab8626e1f523e29ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.840827 11289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d524be8509645a89a75b4bac15f0889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.841418 11746 tablet_bootstrap.cc:700] T 75d0b86635944407b83f5ddfa29fcc4d P 22f3902d01584936b20437606cb86dd1: Time spent opening tablet: real 0.138s user 0.003s sys 0.000s
I20260926 07:35:28.841924 11746 tablet_bootstrap.cc:654] T 75d0b86635944407b83f5ddfa29fcc4d P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:28.841576 11112 catalog_manager.cc:5495] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e reported cstate change: term changed from 0 to 1, leader changed from <none> to e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1). New cstate: current_term: 1 leader_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } health_report { overall_health: UNKNOWN } } }
I20260926 07:35:28.812307 11546 leader_election.cc:302] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:28.813832 11591 tablet_service.cc:1467] Processing CreateTablet for tablet 17871fbd15064daba6e80a34a1b221d3 (DEFAULT_TABLE table=default.Table_8c8e3eb07a9d4b18a589cf8940bbe83f [id=b1dad62530414b119e059c427ccb2130]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.847774 11591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17871fbd15064daba6e80a34a1b221d3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:28.850261 11120 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3e28ca159d8349748812825ac1947c51 consensus state has no leader
I20260926 07:35:28.815414 11582 tablet_service.cc:1467] Processing CreateTablet for tablet f8f025d645eb478f95fa0aef17a86729 (DEFAULT_TABLE table=default.Table_00b32220835a4f1681581a6c9a426f17 [id=072e29bef31847c9beaa569621e5fe7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.852052 11582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8f025d645eb478f95fa0aef17a86729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.784317 11754 raft_consensus.cc:348] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:28.818356 11587 tablet_service.cc:1467] Processing CreateTablet for tablet ed9f7faf55684e3e8512a033aa1443fe (DEFAULT_TABLE table=default.Table_8304ef6caf564378a58911a50bc8d9f7 [id=66cecf81391644ab84399e0f65fa4068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.843592 11969 raft_consensus.cc:482] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260926 07:35:28.844228 11545 leader_election.cc:341] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:28.858544 11716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:35:28.861357 11753 ts_tablet_manager.cc:1918] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:28.861678 11753 log.cc:1196] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/9a255ef54f7e410ca8a779e5d7a118ca
I20260926 07:35:28.862196 11587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed9f7faf55684e3e8512a033aa1443fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.862327 11753 ts_tablet_manager.cc:1939] T 9a255ef54f7e410ca8a779e5d7a118ca P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:28.872552 11716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:35:28.873153 11716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:35:28.876250 11969 raft_consensus.cc:504] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:28.878033 11969 leader_election.cc:288] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:28.880890 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1620b507f07c446c8c1811895a097409" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:28.881445 11463 raft_consensus.cc:3037] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
W20260926 07:35:28.884214 11546 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.1:41829, user_credentials={real_user=slave}} blocked reactor thread for 71989.1us
W20260926 07:35:28.885702 11546 leader_election.cc:341] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Not found: Tablet not found: afa901cdc6a84220bf36ee669a955b72
W20260926 07:35:28.886564 11546 leader_election.cc:341] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Not found: Tablet not found: 1620b507f07c446c8c1811895a097409
I20260926 07:35:28.856703 11754 raft_consensus.cc:374] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:28.846748 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 69) took 2769 ms. Trace:
I20260926 07:35:28.887930 11754 raft_consensus.cc:729] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:28.894323 11754 consensus_queue.cc:260] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:28.823706 11963 raft_consensus.cc:504] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:28.900729 11963 leader_election.cc:288] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Requested vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:28.893002 11969 raft_consensus.cc:2781] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 07:35:28.901656 11969 raft_consensus.cc:482] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:28.902009 11969 raft_consensus.cc:3037] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Advancing to term 2
I20260926 07:35:28.905942 11115 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet c81a59361a6e42b7962b3e027c4a087f
I20260926 07:35:28.906347 11756 tablet_bootstrap.cc:654] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:28.906665 11753 tablet_replica.cc:330] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:28.907402 11753 raft_consensus.cc:2229] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:28.907799 11753 raft_consensus.cc:2258] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:28.907843 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "509f607b089c44bc9c3f80ed59a68c95" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:28.908449 11315 raft_consensus.cc:2454] T 509f607b089c44bc9c3f80ed59a68c95 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 0.
I20260926 07:35:28.909988 11752 ts_tablet_manager.cc:1918] T 067ae4ea43f54e88874d93b4393dae27 P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:28.918927 11754 ts_tablet_manager.cc:1428] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.203s user 0.005s sys 0.000s
I20260926 07:35:28.919781 11754 tablet_bootstrap.cc:492] T d6fc6178f1e54c3984a20873864233cd P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:28.922024 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet 9a255ef54f7e410ca8a779e5d7a118ca (table default.Table_836a28372ec4439d98b1583a9b50d2c1 [id=800a96cbd0b34639bbe7ad317a640053]) successfully deleted
I20260926 07:35:28.923745 11292 tablet_service.cc:1514] Processing DeleteTablet for tablet c81a59361a6e42b7962b3e027c4a087f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:28 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:28.924980 11286 tablet_service.cc:1467] Processing CreateTablet for tablet 25b8b701563344bbbc986db2b0f39b9b (DEFAULT_TABLE table=default.Table_74db930591fe400780cd8ffcfb3ae89d [id=00d642339d644c9688b58c4a32af25c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.929371 11286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25b8b701563344bbbc986db2b0f39b9b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:28.924829 11545 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.2:42379, user_credentials={real_user=slave}} blocked reactor thread for 80770.7us
I20260926 07:35:28.928440 11753 ts_tablet_manager.cc:1905] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 07:35:28.934011 11545 leader_election.cc:341] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Not found: Tablet not found: acc5568dee9b473f8f5f1f8bc3b5504e
I20260926 07:35:28.926153 11290 tablet_service.cc:1467] Processing CreateTablet for tablet 6c42b518ef04429aa6b4e8f5deee05c8 (DEFAULT_TABLE table=default.Table_d94c4858b83943018a693428b6e8a578 [id=0ebbd3b8ecd242c2a21e0630227a01c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.929668 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e1cea4523a64428aac03ca0bdf708c7" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:28.934599 11315 raft_consensus.cc:3037] T 1e1cea4523a64428aac03ca0bdf708c7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:28.926167 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a55e54d492214039b0294f954b5ef390" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:28.935124 11609 raft_consensus.cc:2454] T a55e54d492214039b0294f954b5ef390 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 0.
I20260926 07:35:28.928905 11546 leader_election.cc:302] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:28.891609 11871 rpcz_store.cc:276] 0926 07:35:26.077022 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.079939 (+ 2917us) service_pool.cc:227] Handling call
0926 07:35:28.093115 (+2013176us) ts_tablet_manager.cc:1145] Deleting tablet 50d0e4cd192c445bb7c82d1a3c4a53d1
0926 07:35:28.093166 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:28.627226 (+534060us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:28.809114 (+181888us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:28.846682 (+ 37568us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2010141}
I20260926 07:35:28.940246 11871 tablet_replica.cc:330] T aa02661e7531402687f03e86dbc19418 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:28.940887 11315 raft_consensus.cc:2454] T 1e1cea4523a64428aac03ca0bdf708c7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 1.
I20260926 07:35:28.941135 11871 raft_consensus.cc:2229] T aa02661e7531402687f03e86dbc19418 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:28.941996 11290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c42b518ef04429aa6b4e8f5deee05c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.942006 11871 raft_consensus.cc:2258] T aa02661e7531402687f03e86dbc19418 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:28.944639 11249 leader_election.cc:302] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1
I20260926 07:35:28.944772 11969 raft_consensus.cc:504] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:28.945667 11979 raft_consensus.cc:2781] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:28.945986 11979 raft_consensus.cc:482] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:28.946267 11979 raft_consensus.cc:3037] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:28.955962 11115 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_0d113826d96240549bc980febd8776e0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:28.958056 11115 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0d113826d96240549bc980febd8776e0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:28.958848 11907 raft_consensus.cc:2781] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:28.959251 11907 raft_consensus.cc:482] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:28.959617 11907 raft_consensus.cc:3037] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:28.965415 11969 leader_election.cc:288] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 2 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:28.971040 11979 raft_consensus.cc:504] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:28.973074 11118 master_service.cc:946] ReplaceTablet: received request to replace tablet 5cd482fb159d418291cb02bc9138b076 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:28.973683 11979 leader_election.cc:288] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:28.986173 11112 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_74db930591fe400780cd8ffcfb3ae89d" } modify_external_catalogs: true
I20260926 07:35:28.953889 11445 tablet_service.cc:1467] Processing CreateTablet for tablet 6c42b518ef04429aa6b4e8f5deee05c8 (DEFAULT_TABLE table=default.Table_d94c4858b83943018a693428b6e8a578 [id=0ebbd3b8ecd242c2a21e0630227a01c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.990024 11445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c42b518ef04429aa6b4e8f5deee05c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.954886 11442 tablet_service.cc:1467] Processing CreateTablet for tablet 25b8b701563344bbbc986db2b0f39b9b (DEFAULT_TABLE table=default.Table_74db930591fe400780cd8ffcfb3ae89d [id=00d642339d644c9688b58c4a32af25c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:28.992878 11442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25b8b701563344bbbc986db2b0f39b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:28.975879 11969 consensus_queue.cc:1035] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 07:35:28.994041 11113 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_8b28cfa8ea704f998eae576de3ee6c92" } new_table_name: "default.Table_b49978de02e64080bb6633a3b7c7f7ca" modify_external_catalogs: true
I20260926 07:35:28.955467 11440 tablet_service.cc:1514] Processing DeleteTablet for tablet c81a59361a6e42b7962b3e027c4a087f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:28 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:28.979871 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a55e54d492214039b0294f954b5ef390" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:28.988080 11463 raft_consensus.cc:2454] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 1.
I20260926 07:35:29.005556 11112 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_74db930591fe400780cd8ffcfb3ae89d" } modify_external_catalogs: true
W20260926 07:35:28.935941 11545 leader_election.cc:341] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Not found: Tablet not found: afa901cdc6a84220bf36ee669a955b72
I20260926 07:35:29.007364 11545 leader_election.cc:302] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
W20260926 07:35:29.009831 11247 leader_election.cc:341] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:29.010728 11117 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 640d21586bad4070957cff61bd174fd2
I20260926 07:35:29.011997 11117 catalog_manager.cc:6565] ReplaceTablet: tablet 640d21586bad4070957cff61bd174fd2 deleted and replaced by tablet 6d54a5c32da5444ca0a2939461f3a0f6
I20260926 07:35:28.975991 11546 leader_election.cc:302] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:29.014133 11292 tablet_service.cc:1514] Processing DeleteTablet for tablet 55ad1d96e2314d1eb0a717a3e9ff31bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:29.014505 11969 raft_consensus.cc:2781] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:29.016395 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a55e54d492214039b0294f954b5ef390" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:29.014933 11969 raft_consensus.cc:686] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Becoming Leader. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Running, Role: LEADER
I20260926 07:35:29.016912 11609 raft_consensus.cc:3037] T a55e54d492214039b0294f954b5ef390 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:29.017410 11288 tablet_service.cc:1514] Processing DeleteTablet for tablet 640d21586bad4070957cff61bd174fd2 with delete_type TABLET_DATA_DELETED (replaced by tablet 6d54a5c32da5444ca0a2939461f3a0f6) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:29.020784 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acc5568dee9b473f8f5f1f8bc3b5504e" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:29.017262 11969 consensus_queue.cc:237] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.025887 11435 tablet_service.cc:1514] Processing DeleteTablet for tablet 640d21586bad4070957cff61bd174fd2 with delete_type TABLET_DATA_DELETED (replaced by tablet 6d54a5c32da5444ca0a2939461f3a0f6) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:29.028066 11970 raft_consensus.cc:2726] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:35:29.028532 11545 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.2:42379, user_credentials={real_user=slave}} blocked reactor thread for 92614.4us
I20260926 07:35:29.030114 11609 raft_consensus.cc:2454] T a55e54d492214039b0294f954b5ef390 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 1.
I20260926 07:35:29.037068 11970 consensus_queue.cc:1035] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 07:35:29.036465 11546 leader_election.cc:341] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: Tablet not RUNNING: STOPPED
I20260926 07:35:29.049831 11746 tablet_bootstrap.cc:492] T 75d0b86635944407b83f5ddfa29fcc4d P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:29.051065 11746 ts_tablet_manager.cc:1397] T 75d0b86635944407b83f5ddfa29fcc4d P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.352s user 0.011s sys 0.006s
I20260926 07:35:29.051304 11249 leader_election.cc:302] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1
I20260926 07:35:29.052862 11979 raft_consensus.cc:2781] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:29.053768 11979 raft_consensus.cc:686] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Becoming Leader. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Running, Role: LEADER
I20260926 07:35:29.054507 11979 consensus_queue.cc:237] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.056602 11545 leader_election.cc:302] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters: e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:29.011883 11753 ts_tablet_manager.cc:1918] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:29.055289 11746 raft_consensus.cc:348] T 75d0b86635944407b83f5ddfa29fcc4d P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:29.057350 11970 raft_consensus.cc:2781] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:29.057534 11753 log.cc:1196] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/acc5568dee9b473f8f5f1f8bc3b5504e
I20260926 07:35:29.057873 11970 raft_consensus.cc:686] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Becoming Leader. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Running, Role: LEADER
I20260926 07:35:29.058187 11753 ts_tablet_manager.cc:1939] T acc5568dee9b473f8f5f1f8bc3b5504e P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:29.058570 11970 consensus_queue.cc:237] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:29.060233 11746 raft_consensus.cc:374] T 75d0b86635944407b83f5ddfa29fcc4d P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:29.060497 11746 raft_consensus.cc:729] T 75d0b86635944407b83f5ddfa29fcc4d P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:29.060151 11117 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_19dc9207bc6249c98efc072fe24ba570"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:29.061905 11746 consensus_queue.cc:260] T 75d0b86635944407b83f5ddfa29fcc4d P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:29.064873 11120 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_d94c4858b83943018a693428b6e8a578" } new_table_name: "default.Table_af06af7ab1b64367bfd384a03f121eda" modify_external_catalogs: true
I20260926 07:35:29.070158 11118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 5cd482fb159d418291cb02bc9138b076
I20260926 07:35:29.072832 11118 catalog_manager.cc:6565] ReplaceTablet: tablet 5cd482fb159d418291cb02bc9138b076 deleted and replaced by tablet 393da59b3671473ca9714d3406dd1388
I20260926 07:35:28.944010 11871 ts_tablet_manager.cc:1905] T aa02661e7531402687f03e86dbc19418 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:29.074922 11752 log.cc:1196] T 067ae4ea43f54e88874d93b4393dae27 P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/067ae4ea43f54e88874d93b4393dae27
I20260926 07:35:29.075476 11290 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cd482fb159d418291cb02bc9138b076 with delete_type TABLET_DATA_DELETED (replaced by tablet 393da59b3671473ca9714d3406dd1388) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:29.074889 11116 catalog_manager.cc:5495] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6070e9e3d1234a9d83e49431416afade (127.10.171.3). New cstate: current_term: 1 leader_uuid: "6070e9e3d1234a9d83e49431416afade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } }
I20260926 07:35:29.076210 11752 ts_tablet_manager.cc:1939] T 067ae4ea43f54e88874d93b4393dae27 P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:29.076793 11116 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet a0bdf6a4656e4217b0a7013fddc4cca8 (table default.Table_c7fb27daeba04d90bc0cf0c7ac07fe56 [id=3ce63f9fb5ad4b82ba2fbbf2e3678271]). Expected version 1 got 0
I20260926 07:35:29.078732 11435 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cd482fb159d418291cb02bc9138b076 with delete_type TABLET_DATA_DELETED (replaced by tablet 393da59b3671473ca9714d3406dd1388) from {username='slave'} at 127.0.0.1:40244
W20260926 07:35:29.079128 11117 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_19dc9207bc6249c98efc072fe24ba570 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:29.082515 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet acc5568dee9b473f8f5f1f8bc3b5504e (table default.Table_23c2afadc1164dfab1f47d7c0b373e5f [id=6806510fcf2d4b17960903772e08d49b]) successfully deleted
W20260926 07:35:29.083539 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 4239f5d48512403bb711ece401f0756b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4239f5d48512403bb711ece401f0756b already in progress: creating tablet
W20260926 07:35:29.084563 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 04be706330924282961054c5835cec73 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 04be706330924282961054c5835cec73 already in progress: creating tablet
I20260926 07:35:29.085793 11754 tablet_bootstrap.cc:654] T d6fc6178f1e54c3984a20873864233cd P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:29.086861 11753 tablet_replica.cc:330] T e06ed286144941269b9a0c0c60bff0ce P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:29.087560 11753 raft_consensus.cc:2229] T e06ed286144941269b9a0c0c60bff0ce P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:29.087970 11753 raft_consensus.cc:2258] T e06ed286144941269b9a0c0c60bff0ce P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:29.097645 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 65) took 3456 ms. Trace:
I20260926 07:35:29.099876 11753 ts_tablet_manager.cc:1905] T e06ed286144941269b9a0c0c60bff0ce P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:29.109148 11756 tablet_bootstrap.cc:492] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:29.113657 11112 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 25b8b701563344bbbc986db2b0f39b9b
I20260926 07:35:29.120615 11583 tablet_service.cc:1467] Processing CreateTablet for tablet 6d524be8509645a89a75b4bac15f0889 (DEFAULT_TABLE table=default.Table_f6f34cb43e5c462aa22636e6daaf274e [id=9fae0d37b0f8491ab8626e1f523e29ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:29.121313 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 50d0e4cd192c445bb7c82d1a3c4a53d1 (table default.Table_b8d5ad40dd0b48609fb6831f16cd7fb6 [id=31cfe82ee0424be9bb920fa81e2a22f0]) successfully deleted
I20260926 07:35:29.125185 11590 tablet_service.cc:1467] Processing CreateTablet for tablet 21919d22b1d24546911d36384b01d9cc (DEFAULT_TABLE table=default.Table_4eb22a54d0df4f808fe85886a5809ab7 [id=cab439115f3c46d089824983fe26d0eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:29.130736 11582 tablet_service.cc:1467] Processing CreateTablet for tablet 6c42b518ef04429aa6b4e8f5deee05c8 (DEFAULT_TABLE table=default.Table_d94c4858b83943018a693428b6e8a578 [id=0ebbd3b8ecd242c2a21e0630227a01c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:29.133222 11589 tablet_service.cc:1467] Processing CreateTablet for tablet 25b8b701563344bbbc986db2b0f39b9b (DEFAULT_TABLE table=default.Table_74db930591fe400780cd8ffcfb3ae89d [id=00d642339d644c9688b58c4a32af25c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:29.136431 11580 tablet_service.cc:1514] Processing DeleteTablet for tablet 25b8b701563344bbbc986db2b0f39b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:29 UTC) from {username='slave'} at 127.0.0.1:35244
W20260926 07:35:29.137207 11545 leader_election.cc:341] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Not found: Tablet not found: acc5568dee9b473f8f5f1f8bc3b5504e
I20260926 07:35:29.137640 11545 leader_election.cc:302] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:29.149379 11969 raft_consensus.cc:2726] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260926 07:35:29.139297 11999 log.cc:924] Time spent T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade: Append to log took a long time: real 0.058s user 0.000s sys 0.000s
I20260926 07:35:29.139230 11587 tablet_service.cc:1514] Processing DeleteTablet for tablet c81a59361a6e42b7962b3e027c4a087f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:28 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:29.139689 11588 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cd482fb159d418291cb02bc9138b076 with delete_type TABLET_DATA_DELETED (replaced by tablet 393da59b3671473ca9714d3406dd1388) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:29.140089 11591 tablet_service.cc:1514] Processing DeleteTablet for tablet 640d21586bad4070957cff61bd174fd2 with delete_type TABLET_DATA_DELETED (replaced by tablet 6d54a5c32da5444ca0a2939461f3a0f6) from {username='slave'} at 127.0.0.1:35244
W20260926 07:35:29.158963 11104 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet f31a069dc888487ebbbb1cb91e886ad0 (table default.Table_ba08930510174e65840452201a15f719 [id=47d4200172794cc78a6f386fee4b15b4], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet f31a069dc888487ebbbb1cb91e886ad0 consensus state has no leader
I20260926 07:35:29.025367 11907 raft_consensus.cc:504] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.118839 11286 tablet_service.cc:1514] Processing DeleteTablet for tablet 25b8b701563344bbbc986db2b0f39b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:29 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:29.165187 11756 ts_tablet_manager.cc:1397] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.330s user 0.014s sys 0.001s
I20260926 07:35:29.165583 11907 leader_election.cc:288] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Requested vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:29.109760 11746 ts_tablet_manager.cc:1428] T 75d0b86635944407b83f5ddfa29fcc4d P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.058s user 0.003s sys 0.002s
I20260926 07:35:29.117463 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "509f607b089c44bc9c3f80ed59a68c95" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:29.124326 11464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e1cea4523a64428aac03ca0bdf708c7" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:29.169135 11119 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_e9004884cb974d669eae904cc826d9fb" } modify_external_catalogs: true
I20260926 07:35:29.169620 11119 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_e9004884cb974d669eae904cc826d9fb" } modify_external_catalogs: true
I20260926 07:35:29.170351 11746 tablet_bootstrap.cc:492] T 1e1cea4523a64428aac03ca0bdf708c7 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:29.175410 11746 tablet_bootstrap.cc:654] T 1e1cea4523a64428aac03ca0bdf708c7 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:29.178288 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_c24b830c62a24ff989bceed92112ed52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:29.168015 11118 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_c19ab6c510e64f6bb846aa3dc6f1c2fe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:29.184015 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d524be8509645a89a75b4bac15f0889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:29.185341 11589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25b8b701563344bbbc986db2b0f39b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:29.186583 11582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c42b518ef04429aa6b4e8f5deee05c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:29.187724 11590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21919d22b1d24546911d36384b01d9cc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:29.180104 11113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c24b830c62a24ff989bceed92112ed52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:35:29.184247 11118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c19ab6c510e64f6bb846aa3dc6f1c2fe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:29.167665 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "509f607b089c44bc9c3f80ed59a68c95" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:29.231633 11315 raft_consensus.cc:3037] T 509f607b089c44bc9c3f80ed59a68c95 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:29.233390 11756 raft_consensus.cc:348] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.234068 11756 raft_consensus.cc:374] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:29.234280 11756 raft_consensus.cc:729] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:29.170948 11753 ts_tablet_manager.cc:1918] T e06ed286144941269b9a0c0c60bff0ce P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:29.236284 11753 log.cc:1196] T e06ed286144941269b9a0c0c60bff0ce P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/e06ed286144941269b9a0c0c60bff0ce
I20260926 07:35:29.236912 11753 ts_tablet_manager.cc:1939] T e06ed286144941269b9a0c0c60bff0ce P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:29.239683 11753 tablet_replica.cc:330] T 1e1cea4523a64428aac03ca0bdf708c7 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:29.240854 11114 master_service.cc:946] ReplaceTablet: received request to replace tablet d6fc6178f1e54c3984a20873864233cd from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:29.241406 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet e06ed286144941269b9a0c0c60bff0ce (table default.Table_22c5272f755c4a659ac71d7a5e1c3a22 [id=bad324bd4f1e469ab9435b7a34c83cc4]) successfully deleted
I20260926 07:35:29.241923 11753 raft_consensus.cc:2229] T 1e1cea4523a64428aac03ca0bdf708c7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus shutting down.
W20260926 07:35:29.242034 11106 catalog_manager.cc:4808] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 14e5da3711fe46c4bb46a7a60f41a581 because the tablet was not found. No further retry: Not found: Tablet not found: 14e5da3711fe46c4bb46a7a60f41a581
W20260926 07:35:29.242600 11106 catalog_manager.cc:4808] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet aa02661e7531402687f03e86dbc19418 because the tablet was not found. No further retry: Not found: Tablet not found: aa02661e7531402687f03e86dbc19418
I20260926 07:35:29.244333 11315 raft_consensus.cc:2454] T 509f607b089c44bc9c3f80ed59a68c95 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 1.
I20260926 07:35:29.242305 11753 raft_consensus.cc:2258] T 1e1cea4523a64428aac03ca0bdf708c7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:29.245265 11546 leader_election.cc:302] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:29.246399 11907 raft_consensus.cc:2781] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:29.246645 11753 ts_tablet_manager.cc:1905] T 1e1cea4523a64428aac03ca0bdf708c7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:29.246784 11907 raft_consensus.cc:686] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Becoming Leader. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Running, Role: LEADER
I20260926 07:35:29.247473 11907 consensus_queue.cc:237] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.261953 12015 tablet.cc:1722] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260926 07:35:29.271811 11120 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6c42b518ef04429aa6b4e8f5deee05c8 consensus state has no leader
I20260926 07:35:29.275563 11979 raft_consensus.cc:482] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:29.276386 11979 raft_consensus.cc:504] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.278009 11120 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_8304288ba9e54e32a49193d760c6e068" } new_table_name: "default.Table_4a4f85ee6205445899cac4b93538c3dc" modify_external_catalogs: true
I20260926 07:35:29.280154 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55ad1d96e2314d1eb0a717a3e9ff31bc" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
W20260926 07:35:29.281409 11249 leader_election.cc:341] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:29.282572 11979 leader_election.cc:288] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:29.177404 11752 rpcz_store.cc:276] 0926 07:35:25.641430 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:25.641583 (+ 153us) service_pool.cc:227] Handling call
0926 07:35:28.792561 (+3150978us) ts_tablet_manager.cc:1145] Deleting tablet 067ae4ea43f54e88874d93b4393dae27
0926 07:35:28.792620 (+ 59us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:28.815345 (+ 22725us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:28.909780 (+ 94435us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:29.097589 (+187809us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":19559,"tablet-delete.queue_time_us":3150374}
I20260926 07:35:29.285686 11753 ts_tablet_manager.cc:1918] T 1e1cea4523a64428aac03ca0bdf708c7 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:29.286020 11753 log.cc:1196] T 1e1cea4523a64428aac03ca0bdf708c7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/1e1cea4523a64428aac03ca0bdf708c7
I20260926 07:35:29.286402 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet 067ae4ea43f54e88874d93b4393dae27 (table default.Table_bc4fc15cb6014a1e86ba2886b2950bae [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b]) successfully deleted
I20260926 07:35:29.287495 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 64) took 3646 ms. Trace:
I20260926 07:35:29.287829 11752 rpcz_store.cc:276] 0926 07:35:25.641108 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:25.652766 (+ 11658us) service_pool.cc:227] Handling call
0926 07:35:29.287118 (+3634352us) ts_tablet_manager.cc:1145] Deleting tablet 509f607b089c44bc9c3f80ed59a68c95
0926 07:35:29.287165 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:29.287448 (+ 283us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3633792}
I20260926 07:35:29.288481 11753 ts_tablet_manager.cc:1939] T 1e1cea4523a64428aac03ca0bdf708c7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:29.288859 11754 tablet_bootstrap.cc:492] T d6fc6178f1e54c3984a20873864233cd P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:29.289296 11754 ts_tablet_manager.cc:1397] T d6fc6178f1e54c3984a20873864233cd P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.370s user 0.008s sys 0.002s
W20260926 07:35:29.289558 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 509f607b089c44bc9c3f80ed59a68c95 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 509f607b089c44bc9c3f80ed59a68c95 already in progress: creating tablet
I20260926 07:35:29.267995 12016 tablet.cc:1722] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 07:35:29.282116 11464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55ad1d96e2314d1eb0a717a3e9ff31bc" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:29.288709 11842 raft_consensus.cc:482] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:29.290432 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet 1e1cea4523a64428aac03ca0bdf708c7 (table default.Table_7846bbd3f8cd4fbbb54ce45d2e3ce6b9 [id=41849849f4ae4a9396ccda32d995e21a]) successfully deleted
I20260926 07:35:29.290788 11842 raft_consensus.cc:504] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:29.292408 11753 tablet_replica.cc:330] T 509f607b089c44bc9c3f80ed59a68c95 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:29.292937 11842 leader_election.cc:288] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
W20260926 07:35:29.294045 11401 leader_election.cc:341] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Not found: Tablet not found: e06ed286144941269b9a0c0c60bff0ce
W20260926 07:35:29.294173 11247 leader_election.cc:341] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:29.294515 11247 leader_election.cc:302] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade
I20260926 07:35:29.295186 11979 raft_consensus.cc:2726] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:29.294946 11753 raft_consensus.cc:2229] T 509f607b089c44bc9c3f80ed59a68c95 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:29.296212 11753 raft_consensus.cc:2258] T 509f607b089c44bc9c3f80ed59a68c95 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:29.296141 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06ed286144941269b9a0c0c60bff0ce" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:29.296911 11754 raft_consensus.cc:348] T d6fc6178f1e54c3984a20873864233cd P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.297509 11402 leader_election.cc:302] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 22f3902d01584936b20437606cb86dd1; no voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:29.297699 11753 ts_tablet_manager.cc:1905] T 509f607b089c44bc9c3f80ed59a68c95 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:29.298380 11842 raft_consensus.cc:2726] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:29.260025 11462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "509f607b089c44bc9c3f80ed59a68c95" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:29.302165 11754 raft_consensus.cc:374] T d6fc6178f1e54c3984a20873864233cd P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:29.304975 11754 raft_consensus.cc:729] T d6fc6178f1e54c3984a20873864233cd P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
W20260926 07:35:29.256938 11545 leader_election.cc:341] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:29.308012 11445 tablet_service.cc:1514] Processing DeleteTablet for tablet 25b8b701563344bbbc986db2b0f39b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:29 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:29.306334 11754 consensus_queue.cc:260] T d6fc6178f1e54c3984a20873864233cd P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.309006 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 67) took 3401 ms. Trace:
I20260926 07:35:29.309274 11752 rpcz_store.cc:276] 0926 07:35:25.907559 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:25.907715 (+ 156us) service_pool.cc:227] Handling call
0926 07:35:29.308604 (+3400889us) ts_tablet_manager.cc:1145] Deleting tablet a8b629a4aba045b697e33bbebecb05b7
0926 07:35:29.308652 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:29.308950 (+ 298us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3400316}
W20260926 07:35:29.307119 11545 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.2:42379, user_credentials={real_user=slave}} blocked reactor thread for 50216.6us
W20260926 07:35:29.310434 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet a8b629a4aba045b697e33bbebecb05b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a8b629a4aba045b697e33bbebecb05b7 already in progress: creating tablet
I20260926 07:35:29.312395 11115 master_service.cc:946] ReplaceTablet: received request to replace tablet 55ad1d96e2314d1eb0a717a3e9ff31bc from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:29.314282 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 68) took 3209 ms. Trace:
I20260926 07:35:29.314553 11752 rpcz_store.cc:276] 0926 07:35:26.104833 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.110463 (+ 5630us) service_pool.cc:227] Handling call
0926 07:35:29.313988 (+3203525us) ts_tablet_manager.cc:1145] Deleting tablet 55ad1d96e2314d1eb0a717a3e9ff31bc
0926 07:35:29.314028 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:29.314236 (+ 208us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3202527}
W20260926 07:35:29.315594 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 55ad1d96e2314d1eb0a717a3e9ff31bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55ad1d96e2314d1eb0a717a3e9ff31bc already in progress: creating tablet
I20260926 07:35:29.316802 11111 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet a0bdf6a4656e4217b0a7013fddc4cca8 (table default.Table_c7fb27daeba04d90bc0cf0c7ac07fe56 [id=3ce63f9fb5ad4b82ba2fbbf2e3678271]). Expected version 1 got 0
W20260926 07:35:29.310528 11545 leader_election.cc:341] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:29.323973 11756 consensus_queue.cc:260] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.325807 11754 ts_tablet_manager.cc:1428] T d6fc6178f1e54c3984a20873864233cd P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.036s user 0.005s sys 0.000s
W20260926 07:35:29.330200 11545 leader_election.cc:341] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:29.330775 11871 ts_tablet_manager.cc:1918] T aa02661e7531402687f03e86dbc19418 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:29.334178 11111 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_783e2f568d774b5eb908860af5fa4166" } modify_external_catalogs: true
I20260926 07:35:29.334666 11111 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_783e2f568d774b5eb908860af5fa4166" } modify_external_catalogs: true
I20260926 07:35:29.347126 11756 ts_tablet_manager.cc:1428] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.095s user 0.004s sys 0.000s
I20260926 07:35:29.347946 11756 tablet_bootstrap.cc:492] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:29.352473 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 73) took 2976 ms. Trace:
I20260926 07:35:29.352810 11752 rpcz_store.cc:276] 0926 07:35:26.375823 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.375997 (+ 174us) service_pool.cc:227] Handling call
0926 07:35:29.347019 (+2971022us) ts_tablet_manager.cc:1145] Deleting tablet d6fc6178f1e54c3984a20873864233cd
0926 07:35:29.352168 (+ 5149us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:29.352421 (+ 253us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2970325}
W20260926 07:35:29.353921 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet d6fc6178f1e54c3984a20873864233cd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d6fc6178f1e54c3984a20873864233cd already in progress: creating tablet
I20260926 07:35:29.354709 11752 tablet_replica.cc:330] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:29.355480 11752 raft_consensus.cc:2229] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:29.357600 11112 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet a0bdf6a4656e4217b0a7013fddc4cca8 (table default.Table_c7fb27daeba04d90bc0cf0c7ac07fe56 [id=3ce63f9fb5ad4b82ba2fbbf2e3678271]). Expected version 1 got 0
I20260926 07:35:29.360278 11753 ts_tablet_manager.cc:1918] T 509f607b089c44bc9c3f80ed59a68c95 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:29.360599 11753 log.cc:1196] T 509f607b089c44bc9c3f80ed59a68c95 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/509f607b089c44bc9c3f80ed59a68c95
I20260926 07:35:29.361204 11753 ts_tablet_manager.cc:1939] T 509f607b089c44bc9c3f80ed59a68c95 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:29.368427 11753 tablet_replica.cc:330] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:29.369114 11753 raft_consensus.cc:2229] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:29.369544 11753 raft_consensus.cc:2258] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:29.371110 11753 ts_tablet_manager.cc:1905] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:29.363430 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet 509f607b089c44bc9c3f80ed59a68c95 (table default.Table_65217d931d594a70b45c7d5c486930ba [id=b0949f4718f8482881854a65f89d7421]) successfully deleted
W20260926 07:35:29.373450 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet c81a59361a6e42b7962b3e027c4a087f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c81a59361a6e42b7962b3e027c4a087f already in progress: creating tablet
I20260926 07:35:29.374466 11112 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 4826318c83524f41840e770f33bd3696 (table default.Table_0cf0eeab44174032830f060f976b982e [id=55a6e4b7598043d8bf49512f2542c589]). Expected version 3 got 0
I20260926 07:35:29.335841 11754 tablet_bootstrap.cc:492] T a8b629a4aba045b697e33bbebecb05b7 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:29.381625 11871 log.cc:1196] T aa02661e7531402687f03e86dbc19418 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/aa02661e7531402687f03e86dbc19418
I20260926 07:35:29.382345 11871 ts_tablet_manager.cc:1939] T aa02661e7531402687f03e86dbc19418 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:29.385885 11752 raft_consensus.cc:2258] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:29.387910 11754 tablet_bootstrap.cc:654] T a8b629a4aba045b697e33bbebecb05b7 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:29.363389 11746 tablet_bootstrap.cc:492] T 1e1cea4523a64428aac03ca0bdf708c7 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:29.388394 11746 ts_tablet_manager.cc:1397] T 1e1cea4523a64428aac03ca0bdf708c7 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.218s user 0.010s sys 0.000s
I20260926 07:35:29.394027 11752 ts_tablet_manager.cc:1905] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:29.396533 12020 tablet.cc:1722] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 07:35:29.391651 11746 raft_consensus.cc:348] T 1e1cea4523a64428aac03ca0bdf708c7 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.399040 11746 raft_consensus.cc:374] T 1e1cea4523a64428aac03ca0bdf708c7 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:29.399363 11746 raft_consensus.cc:729] T 1e1cea4523a64428aac03ca0bdf708c7 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:29.401397 11756 tablet_bootstrap.cc:654] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:29.402119 11753 ts_tablet_manager.cc:1918] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:29.399962 11746 consensus_queue.cc:260] T 1e1cea4523a64428aac03ca0bdf708c7 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.405185 11753 log.cc:1196] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/55ad1d96e2314d1eb0a717a3e9ff31bc
I20260926 07:35:29.405507 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 72) took 3327 ms. Trace:
I20260926 07:35:29.405800 11753 ts_tablet_manager.cc:1939] T 55ad1d96e2314d1eb0a717a3e9ff31bc P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:29.414127 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet 55ad1d96e2314d1eb0a717a3e9ff31bc (table default.Table_4d463023afd047528c16b7ee4dabd36f [id=eeb221378f35425eb8d24b815f8a2181]) successfully deleted
W20260926 07:35:29.414857 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 640d21586bad4070957cff61bd174fd2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 640d21586bad4070957cff61bd174fd2 already in progress: creating tablet
W20260926 07:35:29.415517 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 5cd482fb159d418291cb02bc9138b076 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5cd482fb159d418291cb02bc9138b076 already in progress: creating tablet
W20260926 07:35:29.416229 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 25b8b701563344bbbc986db2b0f39b9b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25b8b701563344bbbc986db2b0f39b9b already in progress: creating tablet
I20260926 07:35:29.417656 11119 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet b295ad0f8eaf40778fadae1e3b009bc0
I20260926 07:35:29.407078 11754 tablet_bootstrap.cc:492] T a8b629a4aba045b697e33bbebecb05b7 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:29.418489 11754 ts_tablet_manager.cc:1397] T a8b629a4aba045b697e33bbebecb05b7 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.083s user 0.010s sys 0.001s
I20260926 07:35:29.420666 11754 raft_consensus.cc:348] T a8b629a4aba045b697e33bbebecb05b7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.421274 11754 raft_consensus.cc:374] T a8b629a4aba045b697e33bbebecb05b7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:29.421483 11754 raft_consensus.cc:729] T a8b629a4aba045b697e33bbebecb05b7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:29.422056 11754 consensus_queue.cc:260] T a8b629a4aba045b697e33bbebecb05b7 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.427373 11754 ts_tablet_manager.cc:1428] T a8b629a4aba045b697e33bbebecb05b7 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.009s user 0.004s sys 0.000s
I20260926 07:35:29.428144 11754 tablet_bootstrap.cc:492] T 640d21586bad4070957cff61bd174fd2 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:29.428947 11114 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet a0bdf6a4656e4217b0a7013fddc4cca8 (table default.Table_c7fb27daeba04d90bc0cf0c7ac07fe56 [id=3ce63f9fb5ad4b82ba2fbbf2e3678271]). Expected version 1 got 0
I20260926 07:35:29.428906 11871 rpcz_store.cc:276] 0926 07:35:26.077913 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.086020 (+ 8107us) service_pool.cc:227] Handling call
0926 07:35:28.940040 (+2854020us) ts_tablet_manager.cc:1145] Deleting tablet aa02661e7531402687f03e86dbc19418
0926 07:35:28.940135 (+ 95us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:29.242742 (+302607us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:29.330564 (+ 87822us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:29.405450 (+ 74886us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2853491}
I20260926 07:35:29.429564 11114 catalog_manager.cc:5495] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e reported cstate change: term changed from 0 to 1, leader changed from <none> to e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1). New cstate: current_term: 1 leader_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260926 07:35:29.430663 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet aa02661e7531402687f03e86dbc19418 (table default.Table_17e4f8bb2ea7464fbc563d08108d8325 [id=d91b2ce98c4f439e890cbf47ac6c2d11]) successfully deleted
I20260926 07:35:29.431664 11871 tablet_replica.cc:330] T d6fc6178f1e54c3984a20873864233cd P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:29.432463 11871 raft_consensus.cc:2229] T d6fc6178f1e54c3984a20873864233cd P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:29.432932 11871 raft_consensus.cc:2258] T d6fc6178f1e54c3984a20873864233cd P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:29.415824 11746 ts_tablet_manager.cc:1428] T 1e1cea4523a64428aac03ca0bdf708c7 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.027s user 0.005s sys 0.000s
I20260926 07:35:29.436309 11746 tablet_bootstrap.cc:492] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:29.470455 11871 ts_tablet_manager.cc:1905] T d6fc6178f1e54c3984a20873864233cd P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:29.484176 11115 master_service.cc:946] ReplaceTablet: received request to replace tablet a000cb9928de4459be1e6b1b7a81848a from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:29.470220 11746 tablet_bootstrap.cc:654] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:29.475471 11462 raft_consensus.cc:3037] T 75d0b86635944407b83f5ddfa29fcc4d P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:29.490250 11756 tablet_bootstrap.cc:492] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:29.490760 11756 ts_tablet_manager.cc:1397] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.143s user 0.005s sys 0.005s
I20260926 07:35:29.491008 11754 tablet_bootstrap.cc:654] T 640d21586bad4070957cff61bd174fd2 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:29.493294 11756 raft_consensus.cc:348] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.502269 11979 consensus_queue.cc:1035] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 07:35:29.514889 11756 raft_consensus.cc:374] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:29.518342 11752 ts_tablet_manager.cc:1918] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:29.518723 11752 log.cc:1196] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/f31a069dc888487ebbbb1cb91e886ad0
I20260926 07:35:29.519354 11752 ts_tablet_manager.cc:1939] T f31a069dc888487ebbbb1cb91e886ad0 P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:29.538265 11979 consensus_queue.cc:1035] T 75d0b86635944407b83f5ddfa29fcc4d P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 07:35:29.554248 11119 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_26c01a15748a462e97fc9d5e934e51bf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:29.555866 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 74) took 3149 ms. Trace:
I20260926 07:35:29.556640 11752 rpcz_store.cc:276] 0926 07:35:26.406199 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.406383 (+ 184us) service_pool.cc:227] Handling call
0926 07:35:29.354586 (+2948203us) ts_tablet_manager.cc:1145] Deleting tablet f31a069dc888487ebbbb1cb91e886ad0
0926 07:35:29.354619 (+ 33us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:29.455932 (+101313us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:29.518133 (+ 62201us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:29.555802 (+ 37669us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2947557}
I20260926 07:35:29.557726 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet f31a069dc888487ebbbb1cb91e886ad0 (table default.Table_ba08930510174e65840452201a15f719 [id=47d4200172794cc78a6f386fee4b15b4]) successfully deleted
I20260926 07:35:29.558725 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 83) took 2013 ms. Trace:
I20260926 07:35:29.559012 11752 rpcz_store.cc:276] 0926 07:35:27.545106 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:27.546712 (+ 1606us) service_pool.cc:227] Handling call
0926 07:35:29.558399 (+2011687us) ts_tablet_manager.cc:1145] Deleting tablet c36c759963cb4915afdca7d12109cc6e
0926 07:35:29.558440 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:29.558678 (+ 238us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2004689}
W20260926 07:35:29.560032 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet c36c759963cb4915afdca7d12109cc6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c36c759963cb4915afdca7d12109cc6e already in progress: creating tablet
I20260926 07:35:29.560802 11117 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_5c53cf2e6f714b1db5df09a40f51c11a" } new_table_name: "default.Table_43b629da39c6487a8632545467cd9032" modify_external_catalogs: true
W20260926 07:35:29.555951 11119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_26c01a15748a462e97fc9d5e934e51bf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:29.541424 11756 raft_consensus.cc:729] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:29.584776 11871 ts_tablet_manager.cc:1918] T d6fc6178f1e54c3984a20873864233cd P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:29.594761 11746 tablet_bootstrap.cc:492] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
W20260926 07:35:29.595008 11120 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 52d064b684d0491e91f176d24eddabe1 consensus state has no leader
I20260926 07:35:29.595173 11746 ts_tablet_manager.cc:1397] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.159s user 0.009s sys 0.007s
I20260926 07:35:29.594846 11871 log.cc:1196] T d6fc6178f1e54c3984a20873864233cd P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/d6fc6178f1e54c3984a20873864233cd
I20260926 07:35:29.598630 11752 tablet_replica.cc:330] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:29.599104 11871 ts_tablet_manager.cc:1939] T d6fc6178f1e54c3984a20873864233cd P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:29.599694 11754 tablet_bootstrap.cc:492] T 640d21586bad4070957cff61bd174fd2 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:29.601233 11752 raft_consensus.cc:2229] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:29.601652 11752 raft_consensus.cc:2258] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:29.603191 11752 ts_tablet_manager.cc:1905] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:29.603770 11754 ts_tablet_manager.cc:1397] T 640d21586bad4070957cff61bd174fd2 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.176s user 0.015s sys 0.001s
I20260926 07:35:29.606364 11754 raft_consensus.cc:348] T 640d21586bad4070957cff61bd174fd2 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.607007 11754 raft_consensus.cc:374] T 640d21586bad4070957cff61bd174fd2 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:29.597550 11746 raft_consensus.cc:348] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.610471 11746 raft_consensus.cc:374] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:29.610728 11746 raft_consensus.cc:729] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:29.611900 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet e06ed286144941269b9a0c0c60bff0ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:24 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:29.607249 11754 raft_consensus.cc:729] T 640d21586bad4070957cff61bd174fd2 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:29.615300 12020 tablet.cc:1722] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 07:35:29.618546 11120 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_db30b8ff3c7445d88ef57bc878b2051a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:29.611248 11746 consensus_queue.cc:260] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
W20260926 07:35:29.620432 11120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db30b8ff3c7445d88ef57bc878b2051a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:29.615603 11754 consensus_queue.cc:260] T 640d21586bad4070957cff61bd174fd2 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.626052 11754 ts_tablet_manager.cc:1428] T 640d21586bad4070957cff61bd174fd2 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20260926 07:35:29.626698 11118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_4eb22a54d0df4f808fe85886a5809ab7" } modify_external_catalogs: true
I20260926 07:35:29.627498 11746 ts_tablet_manager.cc:1428] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.032s user 0.005s sys 0.001s
I20260926 07:35:29.627324 11118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_4eb22a54d0df4f808fe85886a5809ab7" } modify_external_catalogs: true
I20260926 07:35:29.628284 11746 tablet_bootstrap.cc:492] T a55e54d492214039b0294f954b5ef390 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:29.632614 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 76) took 3250 ms. Trace:
I20260926 07:35:29.633100 11871 rpcz_store.cc:276] 0926 07:35:26.382206 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.382337 (+ 131us) service_pool.cc:227] Handling call
0926 07:35:29.431504 (+3049167us) ts_tablet_manager.cc:1145] Deleting tablet d6fc6178f1e54c3984a20873864233cd
0926 07:35:29.431561 (+ 57us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:29.504681 (+ 73120us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:29.584571 (+ 79890us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:29.632532 (+ 47961us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3048615}
I20260926 07:35:29.634653 11871 tablet_replica.cc:330] T f31a069dc888487ebbbb1cb91e886ad0 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:29.636869 12016 tablet.cc:1722] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 07:35:29.638435 11754 tablet_bootstrap.cc:492] T ae16af9edff9401589bcf54eeb3e8dfe P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:29.639125 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet d6fc6178f1e54c3984a20873864233cd (table default.Table_051aa34eea804c698486395b4bf2553f [id=f299d82b1a01434690a378d2494fd370]) successfully deleted
I20260926 07:35:29.651904 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet 509f607b089c44bc9c3f80ed59a68c95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:29.661882 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a255ef54f7e410ca8a779e5d7a118ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:23 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:29.570763 11756 consensus_queue.cc:260] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.663137 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet acc5568dee9b473f8f5f1f8bc3b5504e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:23 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:29.665221 11756 ts_tablet_manager.cc:1428] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.174s user 0.006s sys 0.000s
I20260926 07:35:29.671439 11754 tablet_bootstrap.cc:654] T ae16af9edff9401589bcf54eeb3e8dfe P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:29.672483 11746 tablet_bootstrap.cc:654] T a55e54d492214039b0294f954b5ef390 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:29.677261 11111 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 9feaf1ed8ff34928b716fd0b4e41cdde
I20260926 07:35:29.681253 11445 tablet_service.cc:1514] Processing DeleteTablet for tablet 9feaf1ed8ff34928b716fd0b4e41cdde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:29 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:29.681921 11286 tablet_service.cc:1514] Processing DeleteTablet for tablet 9feaf1ed8ff34928b716fd0b4e41cdde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:29 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:29.682724 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet 9feaf1ed8ff34928b716fd0b4e41cdde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:29 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:29.682715 12015 tablet.cc:1722] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 07:35:29.683429 11756 tablet_bootstrap.cc:492] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:29.688218 11752 ts_tablet_manager.cc:1918] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:29.690081 11752 log.cc:1196] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/1620b507f07c446c8c1811895a097409
I20260926 07:35:29.692469 11752 ts_tablet_manager.cc:1939] T 1620b507f07c446c8c1811895a097409 P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
W20260926 07:35:29.692900 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 9feaf1ed8ff34928b716fd0b4e41cdde because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9feaf1ed8ff34928b716fd0b4e41cdde already in progress: creating tablet
I20260926 07:35:29.694329 11754 tablet_bootstrap.cc:492] T ae16af9edff9401589bcf54eeb3e8dfe P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:29.695135 11756 tablet_bootstrap.cc:654] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:29.697472 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 91) took 1480 ms. Trace:
I20260926 07:35:29.694756 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet afa901cdc6a84220bf36ee669a955b72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:24 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:29.700591 11754 ts_tablet_manager.cc:1397] T ae16af9edff9401589bcf54eeb3e8dfe P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.064s user 0.008s sys 0.004s
I20260926 07:35:29.699851 11746 tablet_bootstrap.cc:492] T a55e54d492214039b0294f954b5ef390 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:29.700495 11842 raft_consensus.cc:482] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:29.702529 11842 raft_consensus.cc:504] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.704368 11842 leader_election.cc:288] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:29.705813 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "509f607b089c44bc9c3f80ed59a68c95" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:29.706621 11111 master_service.cc:946] ReplaceTablet: received request to replace tablet a000cb9928de4459be1e6b1b7a81848a from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:29.708024 11746 ts_tablet_manager.cc:1397] T a55e54d492214039b0294f954b5ef390 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.080s user 0.005s sys 0.006s
I20260926 07:35:29.708475 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e1cea4523a64428aac03ca0bdf708c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:24 UTC) from {username='slave'} at 127.0.0.1:35244
W20260926 07:35:29.708621 11401 leader_election.cc:341] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Not found: Tablet not found: 509f607b089c44bc9c3f80ed59a68c95
I20260926 07:35:29.709162 11401 leader_election.cc:302] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 22f3902d01584936b20437606cb86dd1; no voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:29.709846 11842 raft_consensus.cc:2726] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:29.712560 11746 raft_consensus.cc:348] T a55e54d492214039b0294f954b5ef390 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.713274 11746 raft_consensus.cc:374] T a55e54d492214039b0294f954b5ef390 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:29.713533 11746 raft_consensus.cc:729] T a55e54d492214039b0294f954b5ef390 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:29.714872 11746 consensus_queue.cc:260] T a55e54d492214039b0294f954b5ef390 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.716153 11754 raft_consensus.cc:348] T ae16af9edff9401589bcf54eeb3e8dfe P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.713827 11752 rpcz_store.cc:276] 0926 07:35:28.217157 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:28.217278 (+ 121us) service_pool.cc:227] Handling call
0926 07:35:29.598484 (+1381206us) ts_tablet_manager.cc:1145] Deleting tablet 1620b507f07c446c8c1811895a097409
0926 07:35:29.598526 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:29.643056 (+ 44530us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:29.687986 (+ 44930us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:29.697415 (+ 9429us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1380626}
I20260926 07:35:29.717975 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet 1620b507f07c446c8c1811895a097409 (table default.Table_0a660d82b80c40d8a4c41b9abdec566f [id=f23a7ed0439c41149de764ab244ef60b]) successfully deleted
I20260926 07:35:29.718819 11752 tablet_replica.cc:330] T 14e5da3711fe46c4bb46a7a60f41a581 P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:29.724016 11752 raft_consensus.cc:2229] T 14e5da3711fe46c4bb46a7a60f41a581 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:29.729643 11752 raft_consensus.cc:2258] T 14e5da3711fe46c4bb46a7a60f41a581 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:29.718123 11907 consensus_queue.cc:1035] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:29.730079 11754 raft_consensus.cc:374] T ae16af9edff9401589bcf54eeb3e8dfe P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:29.730576 11754 raft_consensus.cc:729] T ae16af9edff9401589bcf54eeb3e8dfe P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:29.732417 11752 ts_tablet_manager.cc:1905] T 14e5da3711fe46c4bb46a7a60f41a581 P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:29.733167 11754 consensus_queue.cc:260] T ae16af9edff9401589bcf54eeb3e8dfe P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.734726 11754 ts_tablet_manager.cc:1428] T ae16af9edff9401589bcf54eeb3e8dfe P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.034s user 0.003s sys 0.002s
I20260926 07:35:29.735634 11116 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet a0bdf6a4656e4217b0a7013fddc4cca8 (table default.Table_c7fb27daeba04d90bc0cf0c7ac07fe56 [id=3ce63f9fb5ad4b82ba2fbbf2e3678271]). Expected version 1 got 0
I20260926 07:35:29.721220 11746 ts_tablet_manager.cc:1428] T a55e54d492214039b0294f954b5ef390 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.012s user 0.004s sys 0.000s
I20260926 07:35:29.739451 11746 tablet_bootstrap.cc:492] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:29.745247 11754 tablet_bootstrap.cc:492] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:29.747835 11907 raft_consensus.cc:482] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:29.748263 11907 raft_consensus.cc:504] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:29.750456 11907 leader_election.cc:288] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:29.751426 11871 raft_consensus.cc:2229] T f31a069dc888487ebbbb1cb91e886ad0 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:29.751950 11871 raft_consensus.cc:2258] T f31a069dc888487ebbbb1cb91e886ad0 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:29.754113 11871 ts_tablet_manager.cc:1905] T f31a069dc888487ebbbb1cb91e886ad0 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:29.721652 12035 raft_consensus.cc:482] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:29.760511 12035 raft_consensus.cc:504] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.765053 11462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7837bc0821a4b68972b663606364955" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:29.766727 12035 leader_election.cc:288] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:29.768854 11462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fc6178f1e54c3984a20873864233cd" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
W20260926 07:35:29.770046 11247 leader_election.cc:341] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:29.771152 11316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7837bc0821a4b68972b663606364955" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:29.777321 11746 tablet_bootstrap.cc:654] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:29.777168 11546 leader_election.cc:341] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
W20260926 07:35:29.778469 11545 leader_election.cc:341] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:29.778894 11545 leader_election.cc:302] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:29.779603 11907 raft_consensus.cc:2726] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:35:29.780413 11249 leader_election.cc:341] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Not found: Tablet not found: d6fc6178f1e54c3984a20873864233cd
I20260926 07:35:29.780807 11249 leader_election.cc:302] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade
I20260926 07:35:29.781755 12035 raft_consensus.cc:2726] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:29.782842 11754 tablet_bootstrap.cc:654] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:29.782881 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_170efaa4a249400e8c1d772a5ac0ec54"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:29.786114 11116 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_e64de6cf413946c79498bb477ac3f3ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:29.788415 11113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_170efaa4a249400e8c1d772a5ac0ec54 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:35:29.791139 11116 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e64de6cf413946c79498bb477ac3f3ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:29.802791 11907 consensus_queue.cc:1035] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:29.815201 11756 tablet_bootstrap.cc:492] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:29.816849 11756 ts_tablet_manager.cc:1397] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.134s user 0.011s sys 0.000s
I20260926 07:35:29.818624 11115 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet a000cb9928de4459be1e6b1b7a81848a
I20260926 07:35:29.822386 11752 ts_tablet_manager.cc:1918] T 14e5da3711fe46c4bb46a7a60f41a581 P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:29.823277 11286 tablet_service.cc:1514] Processing DeleteTablet for tablet a000cb9928de4459be1e6b1b7a81848a with delete_type TABLET_DATA_DELETED (replaced by tablet 89ddc16f2635473da43fe21676a947fa) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:29.824443 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet a000cb9928de4459be1e6b1b7a81848a with delete_type TABLET_DATA_DELETED (replaced by tablet 89ddc16f2635473da43fe21676a947fa) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:29.823971 11753 tablet_replica.cc:330] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:29.824039 11756 raft_consensus.cc:348] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.825641 11753 raft_consensus.cc:2229] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:29.825685 11756 raft_consensus.cc:374] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:29.825980 11756 raft_consensus.cc:729] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:29.826110 11753 raft_consensus.cc:2258] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:29.827105 11752 log.cc:1196] T 14e5da3711fe46c4bb46a7a60f41a581 P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/14e5da3711fe46c4bb46a7a60f41a581
I20260926 07:35:29.827702 11753 ts_tablet_manager.cc:1905] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:29.827811 11752 ts_tablet_manager.cc:1939] T 14e5da3711fe46c4bb46a7a60f41a581 P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:29.830020 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 92) took 1607 ms. Trace:
I20260926 07:35:29.830259 11115 catalog_manager.cc:6565] ReplaceTablet: tablet a000cb9928de4459be1e6b1b7a81848a deleted and replaced by tablet 89ddc16f2635473da43fe21676a947fa
I20260926 07:35:29.830327 11752 rpcz_store.cc:276] 0926 07:35:28.222259 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:28.222361 (+ 102us) service_pool.cc:227] Handling call
0926 07:35:29.718679 (+1496318us) ts_tablet_manager.cc:1145] Deleting tablet 14e5da3711fe46c4bb46a7a60f41a581
0926 07:35:29.718719 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:29.774923 (+ 56204us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:29.822159 (+ 47236us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:29.829945 (+ 7786us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1495768}
I20260926 07:35:29.831429 11752 tablet_replica.cc:330] T 9a255ef54f7e410ca8a779e5d7a118ca P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:29.832496 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet 14e5da3711fe46c4bb46a7a60f41a581 (table default.Table_ff479f29ec0143d6a74b38e25389f1bc [id=2238dbbde77c45cd8689ba10e6dbf472]) successfully deleted
I20260926 07:35:29.833433 11746 tablet_bootstrap.cc:492] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:29.833612 11445 tablet_service.cc:1514] Processing DeleteTablet for tablet a000cb9928de4459be1e6b1b7a81848a with delete_type TABLET_DATA_DELETED (replaced by tablet 89ddc16f2635473da43fe21676a947fa) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:29.833173 11756 consensus_queue.cc:260] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.836598 11754 tablet_bootstrap.cc:492] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:29.837042 11754 ts_tablet_manager.cc:1397] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.094s user 0.007s sys 0.007s
I20260926 07:35:29.839512 11754 raft_consensus.cc:348] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.840065 11115 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_c7fb27daeba04d90bc0cf0c7ac07fe56" } new_table_name: "default.Table_1da003049e4b46eb994aaa3f3094d63c" modify_external_catalogs: true
I20260926 07:35:29.841550 11756 ts_tablet_manager.cc:1428] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
I20260926 07:35:29.841972 11754 raft_consensus.cc:374] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:29.842234 11754 raft_consensus.cc:729] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:29.843500 11756 tablet_bootstrap.cc:492] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:29.845762 11754 consensus_queue.cc:260] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.833810 11746 ts_tablet_manager.cc:1397] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.095s user 0.007s sys 0.001s
I20260926 07:35:29.854328 11753 ts_tablet_manager.cc:1918] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:35:29.854645 11753 log.cc:1196] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/a000cb9928de4459be1e6b1b7a81848a
I20260926 07:35:29.855327 11753 ts_tablet_manager.cc:1939] T a000cb9928de4459be1e6b1b7a81848a P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:29.857973 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet a000cb9928de4459be1e6b1b7a81848a (table default.Table_d0380e9d3f26439bbc91cf476aeae69f [id=19875953f2b6484f934503404af57593]) successfully deleted
I20260926 07:35:29.860671 11754 ts_tablet_manager.cc:1428] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.023s user 0.005s sys 0.001s
I20260926 07:35:29.861555 11752 raft_consensus.cc:2229] T 9a255ef54f7e410ca8a779e5d7a118ca P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:29.862052 11752 raft_consensus.cc:2258] T 9a255ef54f7e410ca8a779e5d7a118ca P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 07:35:29.870174 11546 consensus_peers.cc:487] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1620b507f07c446c8c1811895a097409. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:29.880405 11754 tablet_bootstrap.cc:492] T 0eae53ce278749208492815b918582f3 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:29.880609 11464 raft_consensus.cc:3037] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
W20260926 07:35:29.885388 11546 consensus_peers.cc:487] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e06ed286144941269b9a0c0c60bff0ce. This is attempt 1: this message will repeat every 5th retry.
W20260926 07:35:29.885934 11545 consensus_peers.cc:487] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade -> Peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Couldn't send request to peer 22f3902d01584936b20437606cb86dd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1620b507f07c446c8c1811895a097409. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:29.882795 11752 ts_tablet_manager.cc:1905] T 9a255ef54f7e410ca8a779e5d7a118ca P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:29.894291 11754 tablet_bootstrap.cc:654] T 0eae53ce278749208492815b918582f3 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:29.899307 11756 tablet_bootstrap.cc:654] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:29.907898 11907 consensus_queue.cc:1035] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 07:35:29.910246 11463 raft_consensus.cc:3037] T a55e54d492214039b0294f954b5ef390 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:29.914852 11754 tablet_bootstrap.cc:492] T 0eae53ce278749208492815b918582f3 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:29.915256 11754 ts_tablet_manager.cc:1397] T 0eae53ce278749208492815b918582f3 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.035s user 0.010s sys 0.000s
I20260926 07:35:29.920203 11756 tablet_bootstrap.cc:492] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:29.920588 11756 ts_tablet_manager.cc:1397] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.077s user 0.010s sys 0.002s
I20260926 07:35:29.921041 12035 consensus_queue.cc:1035] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:29.922780 11756 raft_consensus.cc:348] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:29.923414 11756 raft_consensus.cc:374] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:29.923658 11756 raft_consensus.cc:729] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:29.928241 11754 raft_consensus.cc:348] T 0eae53ce278749208492815b918582f3 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:29.928886 11754 raft_consensus.cc:374] T 0eae53ce278749208492815b918582f3 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:29.929145 11754 raft_consensus.cc:729] T 0eae53ce278749208492815b918582f3 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:29.930011 11756 consensus_queue.cc:260] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:29.932369 11752 ts_tablet_manager.cc:1918] T 9a255ef54f7e410ca8a779e5d7a118ca P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:29.932729 11752 log.cc:1196] T 9a255ef54f7e410ca8a779e5d7a118ca P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/9a255ef54f7e410ca8a779e5d7a118ca
I20260926 07:35:29.933369 11752 ts_tablet_manager.cc:1939] T 9a255ef54f7e410ca8a779e5d7a118ca P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:29.935212 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 93) took 1712 ms. Trace:
I20260926 07:35:29.940092 11756 ts_tablet_manager.cc:1428] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.019s user 0.003s sys 0.001s
I20260926 07:35:29.941705 11754 consensus_queue.cc:260] T 0eae53ce278749208492815b918582f3 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:29.942636 11756 tablet_bootstrap.cc:492] T 640d21586bad4070957cff61bd174fd2 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:29.951393 11871 ts_tablet_manager.cc:1918] T f31a069dc888487ebbbb1cb91e886ad0 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:29.951320 11746 raft_consensus.cc:348] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.951956 11746 raft_consensus.cc:374] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:29.952219 11746 raft_consensus.cc:729] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:29.952627 11871 log.cc:1196] T f31a069dc888487ebbbb1cb91e886ad0 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/f31a069dc888487ebbbb1cb91e886ad0
W20260926 07:35:29.948833 11117 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 30df9b4afb414c90af5f4fb0c4582f01 has no consensus state
I20260926 07:35:29.957769 11756 tablet_bootstrap.cc:654] T 640d21586bad4070957cff61bd174fd2 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:29.935520 11752 rpcz_store.cc:276] 0926 07:35:28.222621 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:28.222714 (+ 93us) service_pool.cc:227] Handling call
0926 07:35:29.831293 (+1608579us) ts_tablet_manager.cc:1145] Deleting tablet 9a255ef54f7e410ca8a779e5d7a118ca
0926 07:35:29.831330 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:29.905716 (+ 74386us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:29.932151 (+ 26435us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:29.935155 (+ 3004us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1608138}
I20260926 07:35:29.960094 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 96) took 1694 ms. Trace:
I20260926 07:35:29.960433 11752 rpcz_store.cc:276] 0926 07:35:28.265461 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:28.265601 (+ 140us) service_pool.cc:227] Handling call
0926 07:35:29.959718 (+1694117us) ts_tablet_manager.cc:1145] Deleting tablet 04be706330924282961054c5835cec73
0926 07:35:29.959762 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:29.960043 (+ 281us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1693482}
I20260926 07:35:29.961243 11871 ts_tablet_manager.cc:1939] T f31a069dc888487ebbbb1cb91e886ad0 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:29.951853 11754 ts_tablet_manager.cc:1428] T 0eae53ce278749208492815b918582f3 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.036s user 0.006s sys 0.000s
I20260926 07:35:29.962267 11754 tablet_bootstrap.cc:492] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:29.967540 11754 tablet_bootstrap.cc:654] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:29.978000 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 77) took 3561 ms. Trace:
I20260926 07:35:29.978672 11756 tablet_bootstrap.cc:492] T 640d21586bad4070957cff61bd174fd2 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:29.978405 11871 rpcz_store.cc:276] 0926 07:35:26.416331 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.568414 (+152083us) service_pool.cc:227] Handling call
0926 07:35:29.634407 (+3065993us) ts_tablet_manager.cc:1145] Deleting tablet f31a069dc888487ebbbb1cb91e886ad0
0926 07:35:29.634507 (+ 100us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:29.855636 (+221129us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:29.951179 (+ 95543us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:29.977949 (+ 26770us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":19482,"tablet-delete.queue_time_us":3044593}
I20260926 07:35:29.979084 11756 ts_tablet_manager.cc:1397] T 640d21586bad4070957cff61bd174fd2 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.038s user 0.008s sys 0.002s
I20260926 07:35:29.981459 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet f31a069dc888487ebbbb1cb91e886ad0 (table default.Table_ba08930510174e65840452201a15f719 [id=47d4200172794cc78a6f386fee4b15b4]) successfully deleted
I20260926 07:35:29.982796 11756 raft_consensus.cc:348] T 640d21586bad4070957cff61bd174fd2 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.983491 11756 raft_consensus.cc:374] T 640d21586bad4070957cff61bd174fd2 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:29.983752 11756 raft_consensus.cc:729] T 640d21586bad4070957cff61bd174fd2 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:29.962320 11746 consensus_queue.cc:260] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:29.990751 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet 9a255ef54f7e410ca8a779e5d7a118ca (table default.Table_836a28372ec4439d98b1583a9b50d2c1 [id=800a96cbd0b34639bbe7ad317a640053]) successfully deleted
W20260926 07:35:29.997097 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 04be706330924282961054c5835cec73 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 04be706330924282961054c5835cec73 already in progress: creating tablet
I20260926 07:35:29.980453 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 89) took 2490 ms. Trace:
I20260926 07:35:30.008270 11871 rpcz_store.cc:276] 0926 07:35:27.489700 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:27.490232 (+ 532us) service_pool.cc:227] Handling call
0926 07:35:29.979916 (+2489684us) ts_tablet_manager.cc:1145] Deleting tablet c36c759963cb4915afdca7d12109cc6e
0926 07:35:29.979971 (+ 55us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:29.980365 (+ 394us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2489146}
I20260926 07:35:30.008090 11756 consensus_queue.cc:260] T 640d21586bad4070957cff61bd174fd2 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:30.009636 11871 tablet_replica.cc:330] T 067ae4ea43f54e88874d93b4393dae27 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
W20260926 07:35:30.010430 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet c36c759963cb4915afdca7d12109cc6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c36c759963cb4915afdca7d12109cc6e already in progress: creating tablet
I20260926 07:35:30.011416 11871 raft_consensus.cc:2229] T 067ae4ea43f54e88874d93b4393dae27 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:30.011886 11871 raft_consensus.cc:2258] T 067ae4ea43f54e88874d93b4393dae27 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:30.013754 11871 ts_tablet_manager.cc:1905] T 067ae4ea43f54e88874d93b4393dae27 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:30.013993 11756 ts_tablet_manager.cc:1428] T 640d21586bad4070957cff61bd174fd2 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.033s user 0.000s sys 0.004s
I20260926 07:35:30.014767 11756 tablet_bootstrap.cc:492] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:30.025269 11756 tablet_bootstrap.cc:654] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:30.036710 11756 tablet_bootstrap.cc:492] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:30.037086 11756 ts_tablet_manager.cc:1397] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.007s
I20260926 07:35:30.039124 11756 raft_consensus.cc:348] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:29.998136 11463 raft_consensus.cc:3037] T 1e1cea4523a64428aac03ca0bdf708c7 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:29.961633 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 95) took 1707 ms. Trace:
I20260926 07:35:30.044322 11752 rpcz_store.cc:276] 0926 07:35:28.253903 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:28.264511 (+ 10608us) service_pool.cc:227] Handling call
0926 07:35:29.961319 (+1696808us) ts_tablet_manager.cc:1145] Deleting tablet 4239f5d48512403bb711ece401f0756b
0926 07:35:29.961357 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:29.961588 (+ 231us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1636722}
I20260926 07:35:30.056165 11977 consensus_queue.cc:1035] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:30.058591 11752 tablet_replica.cc:330] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:30.062908 11752 raft_consensus.cc:2229] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:30.063488 11752 raft_consensus.cc:2258] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:30.064669 11977 raft_consensus.cc:482] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:30.065074 11977 raft_consensus.cc:504] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:30.067083 11752 ts_tablet_manager.cc:1905] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:30.069288 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae16af9edff9401589bcf54eeb3e8dfe" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:30.069221 11754 tablet_bootstrap.cc:492] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:30.069922 11609 raft_consensus.cc:2454] T ae16af9edff9401589bcf54eeb3e8dfe P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 0.
I20260926 07:35:30.070092 11754 ts_tablet_manager.cc:1397] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.108s user 0.010s sys 0.000s
I20260926 07:35:30.070703 11977 leader_election.cc:288] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:30.073405 11462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae16af9edff9401589bcf54eeb3e8dfe" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
W20260926 07:35:30.074658 11247 leader_election.cc:341] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:30.078971 11249 leader_election.cc:302] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1
I20260926 07:35:30.080339 11979 raft_consensus.cc:2781] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:30.080650 11979 raft_consensus.cc:482] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:30.080978 11979 raft_consensus.cc:3037] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:30.085518 11754 raft_consensus.cc:348] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:30.090054 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 4239f5d48512403bb711ece401f0756b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4239f5d48512403bb711ece401f0756b already in progress: creating tablet
I20260926 07:35:30.092267 11754 raft_consensus.cc:374] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:30.092185 11970 consensus_queue.cc:1035] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:30.092617 11754 raft_consensus.cc:729] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:30.096094 11117 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_b49978de02e64080bb6633a3b7c7f7ca" } modify_external_catalogs: true
I20260926 07:35:30.097093 11117 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_b49978de02e64080bb6633a3b7c7f7ca" } modify_external_catalogs: true
I20260926 07:35:30.099843 11119 master_service.cc:946] ReplaceTablet: received request to replace tablet c81a59361a6e42b7962b3e027c4a087f from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:30.099988 11754 consensus_queue.cc:260] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.103912 11120 master_service.cc:946] ReplaceTablet: received request to replace tablet 3e28ca159d8349748812825ac1947c51 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:30.087801 11979 raft_consensus.cc:504] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:30.110499 11977 raft_consensus.cc:482] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:30.110921 11977 raft_consensus.cc:504] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:30.113402 11977 leader_election.cc:288] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:30.113368 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b629a4aba045b697e33bbebecb05b7" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:30.113966 11609 raft_consensus.cc:2454] T a8b629a4aba045b697e33bbebecb05b7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 0.
I20260926 07:35:30.114472 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b629a4aba045b697e33bbebecb05b7" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:30.115021 11249 leader_election.cc:302] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
W20260926 07:35:30.115674 11247 leader_election.cc:341] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:30.118110 11746 ts_tablet_manager.cc:1428] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.266s user 0.007s sys 0.000s
I20260926 07:35:30.118355 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae16af9edff9401589bcf54eeb3e8dfe" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:30.118954 11609 raft_consensus.cc:3037] T ae16af9edff9401589bcf54eeb3e8dfe P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:30.119860 11977 raft_consensus.cc:2781] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:30.120201 11977 raft_consensus.cc:482] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:30.120527 11977 raft_consensus.cc:3037] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:30.128885 11754 ts_tablet_manager.cc:1428] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.058s user 0.003s sys 0.002s
I20260926 07:35:30.131512 11754 tablet_bootstrap.cc:492] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:30.133203 11977 raft_consensus.cc:504] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:30.136161 11610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b629a4aba045b697e33bbebecb05b7" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:30.136808 11610 raft_consensus.cc:3037] T a8b629a4aba045b697e33bbebecb05b7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:30.141163 11979 leader_election.cc:288] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:30.039732 11756 raft_consensus.cc:374] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:30.141866 11756 raft_consensus.cc:729] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:30.142525 11756 consensus_queue.cc:260] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:30.143123 11546 consensus_peers.cc:487] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1e1cea4523a64428aac03ca0bdf708c7. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:30.144795 11610 raft_consensus.cc:2454] T a8b629a4aba045b697e33bbebecb05b7 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 1.
I20260926 07:35:30.145076 11970 raft_consensus.cc:482] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:30.145604 11970 raft_consensus.cc:504] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:30.147413 11970 leader_election.cc:288] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:30.148963 11462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55ad1d96e2314d1eb0a717a3e9ff31bc" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:30.150416 11970 raft_consensus.cc:482] T 0eae53ce278749208492815b918582f3 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:30.150933 11970 raft_consensus.cc:504] T 0eae53ce278749208492815b918582f3 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:30.150403 11545 leader_election.cc:341] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:30.128021 11609 raft_consensus.cc:2454] T ae16af9edff9401589bcf54eeb3e8dfe P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 1.
I20260926 07:35:30.154209 11970 leader_election.cc:288] T 0eae53ce278749208492815b918582f3 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
W20260926 07:35:30.155548 11546 leader_election.cc:341] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Not found: Tablet not found: 55ad1d96e2314d1eb0a717a3e9ff31bc
I20260926 07:35:30.155723 11746 tablet_bootstrap.cc:492] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:30.156281 11249 leader_election.cc:302] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:30.157384 11546 leader_election.cc:302] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:30.157568 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eae53ce278749208492815b918582f3" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:30.158349 11907 raft_consensus.cc:2726] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:30.159106 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eae53ce278749208492815b918582f3" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:30.159551 12035 raft_consensus.cc:2781] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:30.171216 11756 ts_tablet_manager.cc:1428] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.134s user 0.005s sys 0.001s
I20260926 07:35:30.176409 11977 leader_election.cc:288] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:30.176785 11756 tablet_bootstrap.cc:492] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:30.176736 12035 raft_consensus.cc:686] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Becoming Leader. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Running, Role: LEADER
I20260926 07:35:30.177835 12035 consensus_queue.cc:237] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:30.182828 11249 leader_election.cc:302] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:30.184230 11977 raft_consensus.cc:2781] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:30.184662 11977 raft_consensus.cc:686] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Becoming Leader. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Running, Role: LEADER
I20260926 07:35:30.185423 11977 consensus_queue.cc:237] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
W20260926 07:35:30.183835 11546 leader_election.cc:341] T 0eae53ce278749208492815b918582f3 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:30.193367 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae16af9edff9401589bcf54eeb3e8dfe" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
W20260926 07:35:30.195030 11247 leader_election.cc:341] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:30.197491 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b629a4aba045b697e33bbebecb05b7" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
W20260926 07:35:30.198482 11247 leader_election.cc:341] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:30.200130 11746 tablet_bootstrap.cc:654] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:30.201798 11545 leader_election.cc:341] T 0eae53ce278749208492815b918582f3 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:30.202185 11545 leader_election.cc:302] T 0eae53ce278749208492815b918582f3 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:30.204279 11907 raft_consensus.cc:2726] T 0eae53ce278749208492815b918582f3 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:30.217763 11119 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_bc4fc15cb6014a1e86ba2886b2950bae" } new_table_name: "default.Table_e8b00b73be7c4818adf8d1e74d731ee6" modify_external_catalogs: true
I20260926 07:35:30.223253 11754 tablet_bootstrap.cc:654] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:30.232342 11871 ts_tablet_manager.cc:1918] T 067ae4ea43f54e88874d93b4393dae27 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:30.232673 11871 log.cc:1196] T 067ae4ea43f54e88874d93b4393dae27 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/067ae4ea43f54e88874d93b4393dae27
I20260926 07:35:30.244817 11842 raft_consensus.cc:482] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:30.245240 11842 raft_consensus.cc:504] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:30.247151 11842 leader_election.cc:288] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:30.252272 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "640d21586bad4070957cff61bd174fd2" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:30.253012 11608 raft_consensus.cc:2454] T 640d21586bad4070957cff61bd174fd2 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
W20260926 07:35:30.253563 11104 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 52d064b684d0491e91f176d24eddabe1 (table default.Table_4a4f85ee6205445899cac4b93538c3dc [id=d19e4ba11fa14fa18c7ae93bf40f7c89], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 52d064b684d0491e91f176d24eddabe1 consensus state has no leader
I20260926 07:35:30.253384 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "640d21586bad4070957cff61bd174fd2" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:30.254004 11315 raft_consensus.cc:2454] T 640d21586bad4070957cff61bd174fd2 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:30.254070 11402 leader_election.cc:302] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:30.255308 11842 raft_consensus.cc:2781] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:30.255671 11842 raft_consensus.cc:482] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:30.256065 11842 raft_consensus.cc:3037] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
W20260926 07:35:30.258999 11546 consensus_peers.cc:487] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a000cb9928de4459be1e6b1b7a81848a. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:30.262650 11871 ts_tablet_manager.cc:1939] T 067ae4ea43f54e88874d93b4393dae27 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:30.264068 11842 raft_consensus.cc:504] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:30.265974 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 97) took 1554 ms. Trace:
I20260926 07:35:30.268049 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "640d21586bad4070957cff61bd174fd2" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:30.269330 11608 raft_consensus.cc:3037] T 640d21586bad4070957cff61bd174fd2 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:30.272583 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "640d21586bad4070957cff61bd174fd2" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:30.273138 11315 raft_consensus.cc:3037] T 640d21586bad4070957cff61bd174fd2 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:30.275703 11608 raft_consensus.cc:2454] T 640d21586bad4070957cff61bd174fd2 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:30.276175 11871 rpcz_store.cc:276] 0926 07:35:28.711919 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:28.770878 (+ 58959us) service_pool.cc:227] Handling call
0926 07:35:30.009503 (+1238625us) ts_tablet_manager.cc:1145] Deleting tablet 067ae4ea43f54e88874d93b4393dae27
0926 07:35:30.009541 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:30.154664 (+145123us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:30.232106 (+ 77442us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:30.265901 (+ 33795us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1237933}
I20260926 07:35:30.277371 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 067ae4ea43f54e88874d93b4393dae27 (table default.Table_bc4fc15cb6014a1e86ba2886b2950bae [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b]) successfully deleted
I20260926 07:35:30.278033 11842 leader_election.cc:288] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:30.279031 11871 tablet_replica.cc:330] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:30.279865 11315 raft_consensus.cc:2454] T 640d21586bad4070957cff61bd174fd2 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:30.281340 11401 leader_election.cc:302] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:30.282210 11842 raft_consensus.cc:2781] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:30.280788 11871 raft_consensus.cc:2229] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:30.285135 12069 raft_consensus.cc:482] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:30.285588 12069 raft_consensus.cc:504] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.286696 11871 raft_consensus.cc:2258] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:30.287420 12069 leader_election.cc:288] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:30.291676 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7837bc0821a4b68972b663606364955" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:30.292542 11463 raft_consensus.cc:3037] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:30.296043 11756 tablet_bootstrap.cc:654] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:30.296686 11546 consensus_peers.cc:487] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 509f607b089c44bc9c3f80ed59a68c95. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:30.301680 11970 consensus_queue.cc:1035] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:30.302457 11754 tablet_bootstrap.cc:492] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:30.302960 11754 ts_tablet_manager.cc:1397] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.173s user 0.011s sys 0.001s
I20260926 07:35:30.303350 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_d605de609d19450cbe6cf9af7b9774fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:30.305423 11113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d605de609d19450cbe6cf9af7b9774fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:30.305158 11754 raft_consensus.cc:348] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.306043 11754 raft_consensus.cc:374] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:30.306289 11754 raft_consensus.cc:729] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:30.307417 11871 ts_tablet_manager.cc:1905] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:30.308548 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7837bc0821a4b68972b663606364955" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:30.309262 11608 raft_consensus.cc:2454] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 0.
I20260926 07:35:30.310844 11754 consensus_queue.cc:260] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.318621 11842 raft_consensus.cc:686] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:30.320343 11842 consensus_queue.cc:237] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:30.328994 11752 ts_tablet_manager.cc:1918] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:30.329317 11752 log.cc:1196] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/e06ed286144941269b9a0c0c60bff0ce
I20260926 07:35:30.329962 11752 ts_tablet_manager.cc:1939] T e06ed286144941269b9a0c0c60bff0ce P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:30.331665 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 97) took 1622 ms. Trace:
I20260926 07:35:30.343282 11754 ts_tablet_manager.cc:1428] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.040s user 0.005s sys 0.000s
I20260926 07:35:30.358911 11754 tablet_bootstrap.cc:492] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:30.378964 11116 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_81f434ad83d24e2b9abc65b59e28381e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:30.383973 11116 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_81f434ad83d24e2b9abc65b59e28381e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:30.411933 11754 tablet_bootstrap.cc:654] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:30.297588 11247 leader_election.cc:341] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:30.431263 11871 ts_tablet_manager.cc:1918] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:30.431624 11871 log.cc:1196] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/14e5da3711fe46c4bb46a7a60f41a581
W20260926 07:35:30.431663 11247 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.2:42379, user_credentials={real_user=slave}} blocked reactor thread for 134091us
I20260926 07:35:30.319451 12035 raft_consensus.cc:482] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:30.433094 12035 raft_consensus.cc:504] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.435200 12035 leader_election.cc:288] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:30.438453 11754 tablet_bootstrap.cc:492] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:30.438866 11754 ts_tablet_manager.cc:1397] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.080s user 0.004s sys 0.004s
I20260926 07:35:30.442317 11754 raft_consensus.cc:348] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.442960 11754 raft_consensus.cc:374] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:30.443192 11754 raft_consensus.cc:729] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:30.444484 11871 ts_tablet_manager.cc:1939] T 14e5da3711fe46c4bb46a7a60f41a581 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:30.446710 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 98) took 1674 ms. Trace:
I20260926 07:35:30.448592 11249 leader_election.cc:302] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1
W20260926 07:35:30.449404 11249 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.3:45127, user_credentials={real_user=slave}} blocked reactor thread for 128608us
I20260926 07:35:30.450750 12035 raft_consensus.cc:2781] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:30.450891 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04be706330924282961054c5835cec73" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:30.451069 12035 raft_consensus.cc:482] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:30.451373 12035 raft_consensus.cc:3037] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:30.443972 11754 consensus_queue.cc:260] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:30.451990 11249 leader_election.cc:341] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:30.452482 11871 rpcz_store.cc:276] 0926 07:35:28.771947 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:28.772040 (+ 93us) service_pool.cc:227] Handling call
0926 07:35:30.278884 (+1506844us) ts_tablet_manager.cc:1145] Deleting tablet 14e5da3711fe46c4bb46a7a60f41a581
0926 07:35:30.278921 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:30.370081 (+ 91160us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:30.431030 (+ 60949us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:30.446653 (+ 15623us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1506339}
I20260926 07:35:30.453527 11871 tablet_replica.cc:330] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:30.453791 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 14e5da3711fe46c4bb46a7a60f41a581 (table default.Table_ff479f29ec0143d6a74b38e25389f1bc [id=2238dbbde77c45cd8689ba10e6dbf472]) successfully deleted
I20260926 07:35:30.454164 11871 raft_consensus.cc:2229] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:30.335368 11746 tablet_bootstrap.cc:492] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:30.452512 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04be706330924282961054c5835cec73" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:30.457736 11746 ts_tablet_manager.cc:1397] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.302s user 0.008s sys 0.003s
I20260926 07:35:30.331946 11752 rpcz_store.cc:276] 0926 07:35:28.709324 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:28.709633 (+ 309us) service_pool.cc:227] Handling call
0926 07:35:30.045168 (+1335535us) ts_tablet_manager.cc:1145] Deleting tablet e06ed286144941269b9a0c0c60bff0ce
0926 07:35:30.058447 (+ 13279us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:30.171700 (+113253us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:30.328802 (+157102us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:30.331616 (+ 2814us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3390,"tablet-delete.queue_time_us":1335010}
I20260926 07:35:30.459200 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 106) took 1506 ms. Trace:
I20260926 07:35:30.459479 11752 rpcz_store.cc:276] 0926 07:35:28.953110 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:28.955322 (+ 2212us) service_pool.cc:227] Handling call
0926 07:35:30.458902 (+1503580us) ts_tablet_manager.cc:1145] Deleting tablet c81a59361a6e42b7962b3e027c4a087f
0926 07:35:30.458941 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:30.459153 (+ 212us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1462440}
I20260926 07:35:30.460413 11746 raft_consensus.cc:348] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:30.461055 11746 raft_consensus.cc:374] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:30.461297 11746 raft_consensus.cc:729] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:30.462280 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet e06ed286144941269b9a0c0c60bff0ce (table default.Table_22c5272f755c4a659ac71d7a5e1c3a22 [id=bad324bd4f1e469ab9435b7a34c83cc4]) successfully deleted
I20260926 07:35:30.462574 11754 ts_tablet_manager.cc:1428] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.023s user 0.007s sys 0.000s
W20260926 07:35:30.462518 11247 leader_election.cc:341] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
W20260926 07:35:30.463038 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet c81a59361a6e42b7962b3e027c4a087f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c81a59361a6e42b7962b3e027c4a087f already in progress: creating tablet
I20260926 07:35:30.462932 11247 leader_election.cc:302] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade
I20260926 07:35:30.463894 11977 raft_consensus.cc:2726] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:30.464722 11746 consensus_queue.cc:260] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:30.466797 11746 ts_tablet_manager.cc:1428] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260926 07:35:30.472741 11746 tablet_bootstrap.cc:492] T b7837bc0821a4b68972b663606364955 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:30.472963 11752 tablet_replica.cc:330] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:30.476696 11752 raft_consensus.cc:2229] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:30.477275 11752 pending_rounds.cc:62] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1: Trying to abort 1 pending ops.
I20260926 07:35:30.477455 11752 pending_rounds.cc:68] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333511701808152576 op_type: NO_OP noop_request { }
I20260926 07:35:30.477749 11752 raft_consensus.cc:2866] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 07:35:30.478062 11752 raft_consensus.cc:2258] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:30.478348 11754 tablet_bootstrap.cc:492] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:30.481421 11871 pending_rounds.cc:62] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade: Trying to abort 1 pending ops.
I20260926 07:35:30.481657 11871 pending_rounds.cc:68] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333511696637198336 op_type: NO_OP noop_request { }
I20260926 07:35:30.482009 11871 raft_consensus.cc:2866] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 07:35:30.482324 11871 raft_consensus.cc:2258] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:30.483986 11871 ts_tablet_manager.cc:1905] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:30.492241 11746 tablet_bootstrap.cc:654] T b7837bc0821a4b68972b663606364955 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:30.502844 11752 ts_tablet_manager.cc:1905] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:30.514468 11754 tablet_bootstrap.cc:654] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:30.516554 11119 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0218d2a727f942e48d770878e6bf4a54 consensus state has no leader
I20260926 07:35:30.521922 11445 tablet_service.cc:1467] Processing CreateTablet for tablet e5d942ed0a7b4f81bb9bfb915b5512f7 (DEFAULT_TABLE table=default.Table_4419f8321f5e4c0ebd5b75bb80a5d133 [id=869ffb6d402043f2969cdea1c84ea926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:30.523025 11445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5d942ed0a7b4f81bb9bfb915b5512f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:30.524612 11442 tablet_service.cc:1467] Processing CreateTablet for tablet 5ee33141350b4c31aae79d2160e79b6f (DEFAULT_TABLE table=default.Table_02974e73af5f4521a34bcb9d2c75a84a [id=295cf894a5064a209ee455ddbdfa528a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:30.525490 11442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ee33141350b4c31aae79d2160e79b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:30.528460 11435 tablet_service.cc:1467] Processing CreateTablet for tablet 30df9b4afb414c90af5f4fb0c4582f01 (DEFAULT_TABLE table=default.Table_43b629da39c6487a8632545467cd9032 [id=156c350dad7246448e3f55e59820e470]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:30.529469 11435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30df9b4afb414c90af5f4fb0c4582f01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:30.531555 11112 catalog_manager.cc:5495] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade reported cstate change: term changed from 0 to 1, leader changed from <none> to 6070e9e3d1234a9d83e49431416afade (127.10.171.3). New cstate: current_term: 1 leader_uuid: "6070e9e3d1234a9d83e49431416afade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } health_report { overall_health: UNKNOWN } } }
I20260926 07:35:30.533425 11112 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 4826318c83524f41840e770f33bd3696 (table default.Table_0cf0eeab44174032830f060f976b982e [id=55a6e4b7598043d8bf49512f2542c589]). Expected version 3 got 0
I20260926 07:35:30.534680 11582 tablet_service.cc:1467] Processing CreateTablet for tablet 5ee33141350b4c31aae79d2160e79b6f (DEFAULT_TABLE table=default.Table_02974e73af5f4521a34bcb9d2c75a84a [id=295cf894a5064a209ee455ddbdfa528a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:30.534991 11290 tablet_service.cc:1467] Processing CreateTablet for tablet 5ee33141350b4c31aae79d2160e79b6f (DEFAULT_TABLE table=default.Table_02974e73af5f4521a34bcb9d2c75a84a [id=295cf894a5064a209ee455ddbdfa528a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:30.535059 11288 tablet_service.cc:1467] Processing CreateTablet for tablet 30df9b4afb414c90af5f4fb0c4582f01 (DEFAULT_TABLE table=default.Table_43b629da39c6487a8632545467cd9032 [id=156c350dad7246448e3f55e59820e470]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:30.536171 11286 tablet_service.cc:1467] Processing CreateTablet for tablet e5d942ed0a7b4f81bb9bfb915b5512f7 (DEFAULT_TABLE table=default.Table_4419f8321f5e4c0ebd5b75bb80a5d133 [id=869ffb6d402043f2969cdea1c84ea926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:30.535995 11290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ee33141350b4c31aae79d2160e79b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:30.537199 11286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5d942ed0a7b4f81bb9bfb915b5512f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:30.537721 11288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30df9b4afb414c90af5f4fb0c4582f01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:30.538626 11583 tablet_service.cc:1467] Processing CreateTablet for tablet e5d942ed0a7b4f81bb9bfb915b5512f7 (DEFAULT_TABLE table=default.Table_4419f8321f5e4c0ebd5b75bb80a5d133 [id=869ffb6d402043f2969cdea1c84ea926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:30.539791 11582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ee33141350b4c31aae79d2160e79b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:30.542305 11590 tablet_service.cc:1467] Processing CreateTablet for tablet 30df9b4afb414c90af5f4fb0c4582f01 (DEFAULT_TABLE table=default.Table_43b629da39c6487a8632545467cd9032 [id=156c350dad7246448e3f55e59820e470]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:30.542791 11292 tablet_service.cc:1467] Processing CreateTablet for tablet c1e934f75d604d66aa12c9c8411d137a (DEFAULT_TABLE table=default.Table_13ddd864a4c2474abb61d8c3b975b8d7 [id=412ff7bc1ddf4fc8a4b1714736493c7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:30.543979 11440 tablet_service.cc:1467] Processing CreateTablet for tablet c1e934f75d604d66aa12c9c8411d137a (DEFAULT_TABLE table=default.Table_13ddd864a4c2474abb61d8c3b975b8d7 [id=412ff7bc1ddf4fc8a4b1714736493c7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:30.547530 11440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1e934f75d604d66aa12c9c8411d137a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:30.548342 11590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30df9b4afb414c90af5f4fb0c4582f01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:30.553841 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5d942ed0a7b4f81bb9bfb915b5512f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:30.554819 11589 tablet_service.cc:1467] Processing CreateTablet for tablet c1e934f75d604d66aa12c9c8411d137a (DEFAULT_TABLE table=default.Table_13ddd864a4c2474abb61d8c3b975b8d7 [id=412ff7bc1ddf4fc8a4b1714736493c7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:30.555685 11589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1e934f75d604d66aa12c9c8411d137a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:30.568085 11292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1e934f75d604d66aa12c9c8411d137a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:30.577195 11113 master_service.cc:946] ReplaceTablet: received request to replace tablet 04180ace5d4548dfbcd226bf229d5c26 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:30.590996 12035 raft_consensus.cc:504] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.602248 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7837bc0821a4b68972b663606364955" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
W20260926 07:35:30.603775 11247 leader_election.cc:341] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:30.605307 11970 raft_consensus.cc:482] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:30.605741 11970 raft_consensus.cc:504] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.607611 12035 leader_election.cc:288] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Requested vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:30.607777 11970 leader_election.cc:288] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:30.609472 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7837bc0821a4b68972b663606364955" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:30.610047 11608 raft_consensus.cc:3037] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:30.613325 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c81a59361a6e42b7962b3e027c4a087f" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:30.613637 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c81a59361a6e42b7962b3e027c4a087f" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
W20260926 07:35:30.614521 11545 leader_election.cc:341] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
W20260926 07:35:30.614925 11546 leader_election.cc:341] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:30.615249 11546 leader_election.cc:302] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:30.616401 11970 raft_consensus.cc:2726] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:30.618571 11608 raft_consensus.cc:2454] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 1.
I20260926 07:35:30.619616 11249 leader_election.cc:302] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1
I20260926 07:35:30.625577 11746 tablet_bootstrap.cc:492] T b7837bc0821a4b68972b663606364955 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:30.626006 11746 ts_tablet_manager.cc:1397] T b7837bc0821a4b68972b663606364955 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.153s user 0.004s sys 0.004s
I20260926 07:35:30.628211 11746 raft_consensus.cc:348] T b7837bc0821a4b68972b663606364955 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.628872 11746 raft_consensus.cc:374] T b7837bc0821a4b68972b663606364955 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:30.629125 11746 raft_consensus.cc:729] T b7837bc0821a4b68972b663606364955 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:30.636673 11116 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_e0dcc535d6e54d679a9444cf6cec1445"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:30.638226 11116 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e0dcc535d6e54d679a9444cf6cec1445 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:30.640303 12035 raft_consensus.cc:2781] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:30.641453 12035 raft_consensus.cc:686] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Becoming Leader. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Running, Role: LEADER
I20260926 07:35:30.642844 12035 consensus_queue.cc:237] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.644948 11756 tablet_bootstrap.cc:492] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:30.645450 11756 ts_tablet_manager.cc:1397] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.469s user 0.010s sys 0.000s
I20260926 07:35:30.645789 11754 tablet_bootstrap.cc:492] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:30.646224 11754 ts_tablet_manager.cc:1397] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.168s user 0.007s sys 0.001s
I20260926 07:35:30.646795 11746 consensus_queue.cc:260] T b7837bc0821a4b68972b663606364955 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.648734 11746 ts_tablet_manager.cc:1428] T b7837bc0821a4b68972b663606364955 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
I20260926 07:35:30.648518 11756 raft_consensus.cc:348] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.653630 11756 raft_consensus.cc:374] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:30.653919 11756 raft_consensus.cc:729] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:30.664937 11746 tablet_bootstrap.cc:492] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:30.667521 11756 consensus_queue.cc:260] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.671633 11746 tablet_bootstrap.cc:654] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:30.676057 11756 ts_tablet_manager.cc:1428] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.030s user 0.004s sys 0.000s
I20260926 07:35:30.681665 11756 tablet_bootstrap.cc:492] T 034d508f0a164f3ea8308ea2d28818c4 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
W20260926 07:35:30.689656 11104 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:37629: connect: Connection refused (error 111) [suppressed 6 similar messages]
I20260926 07:35:30.689950 11746 tablet_bootstrap.cc:492] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:30.697181 11746 ts_tablet_manager.cc:1397] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.032s user 0.005s sys 0.004s
I20260926 07:35:30.699005 11970 raft_consensus.cc:482] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:30.699201 11746 raft_consensus.cc:348] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:30.699829 11746 raft_consensus.cc:374] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:30.699587 11970 raft_consensus.cc:504] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.701309 11970 leader_election.cc:288] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:30.703197 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9feaf1ed8ff34928b716fd0b4e41cdde" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
W20260926 07:35:30.705226 11546 leader_election.cc:341] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:30.706996 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9feaf1ed8ff34928b716fd0b4e41cdde" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
W20260926 07:35:30.708113 11545 leader_election.cc:341] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:30.700065 11746 raft_consensus.cc:729] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:30.708526 11545 leader_election.cc:302] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:30.709508 11970 raft_consensus.cc:2726] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:30.708961 11746 consensus_queue.cc:260] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:30.712373 11746 ts_tablet_manager.cc:1428] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260926 07:35:30.650614 11754 raft_consensus.cc:348] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.722931 11754 raft_consensus.cc:374] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:30.723176 11754 raft_consensus.cc:729] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:30.723974 11754 consensus_queue.cc:260] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.735391 11871 ts_tablet_manager.cc:1918] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:30.735739 11871 log.cc:1196] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/1620b507f07c446c8c1811895a097409
I20260926 07:35:30.739187 11746 tablet_bootstrap.cc:492] T a8b629a4aba045b697e33bbebecb05b7 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:30.758178 11754 ts_tablet_manager.cc:1428] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.112s user 0.007s sys 0.001s
I20260926 07:35:30.767975 11120 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 3e28ca159d8349748812825ac1947c51
I20260926 07:35:30.769279 11120 catalog_manager.cc:6565] ReplaceTablet: tablet 3e28ca159d8349748812825ac1947c51 deleted and replaced by tablet 2dd5175f0ca941f8b3856ea415e2ee68
I20260926 07:35:30.772746 11746 tablet_bootstrap.cc:654] T a8b629a4aba045b697e33bbebecb05b7 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:30.771998 11119 catalog_manager.cc:5495] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e reported cstate change: term changed from 0 to 1, leader changed from <none> to e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1). New cstate: current_term: 1 leader_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260926 07:35:30.773303 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e28ca159d8349748812825ac1947c51 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dd5175f0ca941f8b3856ea415e2ee68) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:30.773653 11119 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet ae16af9edff9401589bcf54eeb3e8dfe (table default.Table_f9178d5e3d934f1f9ef267a782316458 [id=0c18461963f444269787b667b3648709]). Expected version 2 got 0
I20260926 07:35:30.774355 11119 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet 4826318c83524f41840e770f33bd3696 (table default.Table_0cf0eeab44174032830f060f976b982e [id=55a6e4b7598043d8bf49512f2542c589]). Expected version 3 got 0
I20260926 07:35:30.773890 11445 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e28ca159d8349748812825ac1947c51 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dd5175f0ca941f8b3856ea415e2ee68) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:30.776355 11292 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e28ca159d8349748812825ac1947c51 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dd5175f0ca941f8b3856ea415e2ee68) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:30.777279 11114 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: default.Table_b49978de02e64080bb6633a3b7c7f7ca [id=9e879069d99f41cdb271157af1a3fd1e] alter complete (version 2)
W20260926 07:35:30.778671 11114 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.1:34359 (request call id 42) took 1452 ms (client timeout 1000 ms). Trace:
I20260926 07:35:30.779173 11754 tablet_bootstrap.cc:492] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
W20260926 07:35:30.783376 11111 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.2:33317 (request call id 45) took 1010 ms (client timeout 1000 ms). Trace:
I20260926 07:35:30.784581 11871 ts_tablet_manager.cc:1939] T 1620b507f07c446c8c1811895a097409 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:30.786837 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 99) took 2014 ms. Trace:
I20260926 07:35:30.787125 11871 rpcz_store.cc:276] 0926 07:35:28.772881 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:28.772972 (+ 91us) service_pool.cc:227] Handling call
0926 07:35:30.453388 (+1680416us) ts_tablet_manager.cc:1145] Deleting tablet 1620b507f07c446c8c1811895a097409
0926 07:35:30.453428 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:30.661630 (+208202us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:30.735193 (+ 73563us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:30.786788 (+ 51595us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":26522,"tablet-delete.queue_time_us":1679974}
I20260926 07:35:30.790761 11746 tablet_bootstrap.cc:492] T a8b629a4aba045b697e33bbebecb05b7 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:30.791118 11746 ts_tablet_manager.cc:1397] T a8b629a4aba045b697e33bbebecb05b7 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.052s user 0.011s sys 0.000s
I20260926 07:35:30.793457 11445 tablet_service.cc:1514] Processing DeleteTablet for tablet 509f607b089c44bc9c3f80ed59a68c95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:30.794037 11440 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e1cea4523a64428aac03ca0bdf708c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:24 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:30.793186 11746 raft_consensus.cc:348] T a8b629a4aba045b697e33bbebecb05b7 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:30.798623 11746 raft_consensus.cc:374] T a8b629a4aba045b697e33bbebecb05b7 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260926 07:35:30.783646 11111 rpcz_store.cc:269] 0926 07:35:29.772400 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.821166 (+ 48766us) service_pool.cc:227] Handling call
0926 07:35:29.836852 (+ 15686us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812374c3e 00007fb80feabfb4 00007fb80feabf42 00007fb80feabe71 00007fb80feabdec 00007fb80feab0a2 00007fb80e15594c 00007fb80e154d86
0926 07:35:30.783220 (+946368us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":6,"spinlock_wait_cycles":32021376}
I20260926 07:35:30.798934 11746 raft_consensus.cc:729] T a8b629a4aba045b697e33bbebecb05b7 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:30.799705 11746 consensus_queue.cc:260] T a8b629a4aba045b697e33bbebecb05b7 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:30.804684 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 102) took 2013 ms. Trace:
I20260926 07:35:30.805011 11871 rpcz_store.cc:276] 0926 07:35:28.791557 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:28.791643 (+ 86us) service_pool.cc:227] Handling call
0926 07:35:30.788011 (+1996368us) ts_tablet_manager.cc:1145] Deleting tablet 4239f5d48512403bb711ece401f0756b
0926 07:35:30.788047 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:30.804628 (+ 16581us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1995853}
I20260926 07:35:30.805752 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 1620b507f07c446c8c1811895a097409 (table default.Table_0a660d82b80c40d8a4c41b9abdec566f [id=f23a7ed0439c41149de764ab244ef60b]) successfully deleted
I20260926 07:35:30.806123 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 103) took 2013 ms. Trace:
I20260926 07:35:30.806380 11871 rpcz_store.cc:276] 0926 07:35:28.792514 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:28.792617 (+ 103us) service_pool.cc:227] Handling call
0926 07:35:30.805809 (+2013192us) ts_tablet_manager.cc:1145] Deleting tablet 04be706330924282961054c5835cec73
0926 07:35:30.805871 (+ 62us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:30.806077 (+ 206us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2012737}
I20260926 07:35:30.806809 11118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 21919d22b1d24546911d36384b01d9cc
I20260926 07:35:30.807394 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 115) took 1671 ms. Trace:
I20260926 07:35:30.808163 11118 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:44836 (request call id 199) took 1182 ms. Trace:
I20260926 07:35:30.784050 11754 tablet_bootstrap.cc:654] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:30.808494 11118 rpcz_store.cc:276] 0926 07:35:29.626176 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.626326 (+ 150us) service_pool.cc:227] Handling call
0926 07:35:29.627823 (+ 1497us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0926 07:35:29.629842 (+ 2019us) catalog_manager.cc:2752] Modifying in-memory table state
0926 07:35:29.629993 (+ 151us) catalog_manager.cc:2761] Locking tablets
0926 07:35:30.515325 (+885332us) catalog_manager.cc:2775] Removing table and tablets from system table
0926 07:35:30.767232 (+251907us) catalog_manager.cc:2793] Removing table from by-name map
0926 07:35:30.767469 (+ 237us) catalog_manager.cc:2809] Committing in-memory state
0926 07:35:30.792255 (+ 24786us) catalog_manager.cc:2813] Aborting table tasks
0926 07:35:30.808085 (+ 15830us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:30.663297 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:30.663646 (+ 349us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:30.663667 (+ 21us) write_op.cc:435] Acquired schema lock
0926 07:35:30.663679 (+ 12us) tablet.cc:660] Decoding operations
0926 07:35:30.663974 (+ 295us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:30.664009 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:30.664031 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0926 07:35:30.664304 (+ 273us) tablet.cc:699] Row locks acquired
0926 07:35:30.664318 (+ 14us) write_op.cc:260] PREPARE: finished
0926 07:35:30.664426 (+ 108us) write_op.cc:270] Start()
0926 07:35:30.664501 (+ 75us) write_op.cc:276] Timestamp: P: 1790408130664378 usec, L: 0
0926 07:35:30.664516 (+ 15us) op_driver.cc:331] REPLICATION: starting
0926 07:35:30.664944 (+ 428us) log.cc:842] Serialized 728 byte log entry
0926 07:35:30.761832 (+ 96888us) op_driver.cc:444] REPLICATION: finished
0926 07:35:30.764233 (+ 2401us) write_op.cc:301] APPLY: starting
0926 07:35:30.764276 (+ 43us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:30.764375 (+ 99us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:30.764455 (+ 80us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:30.764681 (+ 226us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:30.764798 (+ 117us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0926 07:35:30.764840 (+ 42us) write_op.cc:312] APPLY: finished
0926 07:35:30.764981 (+ 141us) log.cc:842] Serialized 29 byte log entry
0926 07:35:30.765085 (+ 104us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:30.765137 (+ 52us) write_op.cc:454] Released schema lock
0926 07:35:30.765162 (+ 25us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:30.765304 (+ 142us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:30.765327 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2279,"apply.run_cpu_time_us":1261,"apply.run_wall_time_us":1261,"num_ops":2,"prepare.queue_time_us":137631,"prepare.run_cpu_time_us":1871,"prepare.run_wall_time_us":13538,"replication_time_us":97235}]]}
I20260926 07:35:30.812211 11746 ts_tablet_manager.cc:1428] T a8b629a4aba045b697e33bbebecb05b7 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20260926 07:35:30.813325 11746 tablet_bootstrap.cc:492] T ae16af9edff9401589bcf54eeb3e8dfe P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:30.815189 11286 tablet_service.cc:1514] Processing DeleteTablet for tablet 21919d22b1d24546911d36384b01d9cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:29 UTC) from {username='slave'} at 127.0.0.1:52070
W20260926 07:35:30.817145 11545 consensus_peers.cc:487] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade -> Peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Couldn't send request to peer 22f3902d01584936b20437606cb86dd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e06ed286144941269b9a0c0c60bff0ce. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:30.817543 11582 tablet_service.cc:1514] Processing DeleteTablet for tablet 21919d22b1d24546911d36384b01d9cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:29 UTC) from {username='slave'} at 127.0.0.1:35244
W20260926 07:35:30.817762 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 4239f5d48512403bb711ece401f0756b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4239f5d48512403bb711ece401f0756b already in progress: creating tablet
I20260926 07:35:30.819670 11440 tablet_service.cc:1514] Processing DeleteTablet for tablet 21919d22b1d24546911d36384b01d9cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:29 UTC) from {username='slave'} at 127.0.0.1:40244
W20260926 07:35:30.778959 11114 rpcz_store.cc:269] 0926 07:35:29.326637 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.326728 (+ 91us) service_pool.cc:227] Handling call
0926 07:35:29.336675 (+ 9947us) spinlock_profiling.cc:243] Waited 8.58 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812374c3e 00007fb80feabfb4 00007fb80feabf42 00007fb80feabe71 00007fb80feabdec 00007fb80feab0a2 00007fb80e15594c 00007fb80e154d86
0926 07:35:30.778584 (+1441909us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:29.689868 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:29.690281 (+ 413us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:29.690306 (+ 25us) write_op.cc:435] Acquired schema lock
0926 07:35:29.690322 (+ 16us) tablet.cc:660] Decoding operations
0926 07:35:29.690620 (+ 298us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:29.690661 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:29.690692 (+ 31us) tablet.cc:683] Acquiring locks for 1 operations
0926 07:35:29.690841 (+ 149us) tablet.cc:699] Row locks acquired
0926 07:35:29.690858 (+ 17us) write_op.cc:260] PREPARE: finished
0926 07:35:29.690969 (+ 111us) write_op.cc:270] Start()
0926 07:35:29.691051 (+ 82us) write_op.cc:276] Timestamp: P: 1790408129690895 usec, L: 0
0926 07:35:29.691069 (+ 18us) op_driver.cc:331] REPLICATION: starting
0926 07:35:29.691588 (+ 519us) log.cc:842] Serialized 530 byte log entry
0926 07:35:29.728729 (+ 37141us) op_driver.cc:444] REPLICATION: finished
0926 07:35:29.729076 (+ 347us) write_op.cc:301] APPLY: starting
0926 07:35:29.729130 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:29.729609 (+ 479us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:29.729701 (+ 92us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:29.729891 (+ 190us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:29.730066 (+ 175us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:29.730127 (+ 61us) write_op.cc:312] APPLY: finished
0926 07:35:29.730336 (+ 209us) log.cc:842] Serialized 23 byte log entry
0926 07:35:29.730518 (+ 182us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:29.730597 (+ 79us) write_op.cc:454] Released schema lock
0926 07:35:29.730635 (+ 38us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:29.730834 (+ 199us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:29.730865 (+ 31us) write_op.cc:454] Released schema lock
Related trace 'op':
0926 07:35:30.130165 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:30.130911 (+ 746us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:30.130968 (+ 57us) write_op.cc:435] Acquired schema lock
0926 07:35:30.131000 (+ 32us) tablet.cc:660] Decoding operations
0926 07:35:30.131719 (+ 719us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:30.131792 (+ 73us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:30.131841 (+ 49us) tablet.cc:683] Acquiring locks for 1 operations
0926 07:35:30.132085 (+ 244us) tablet.cc:699] Row locks acquired
0926 07:35:30.132112 (+ 27us) write_op.cc:260] PREPARE: finished
0926 07:35:30.145987 (+ 13875us) write_op.cc:270] Start()
0926 07:35:30.146118 (+ 131us) write_op.cc:276] Timestamp: P: 1790408130145881 usec, L: 0
0926 07:35:30.146150 (+ 32us) op_driver.cc:331] REPLICATION: starting
0926 07:35:30.213646 (+ 67496us) spinlock_profiling.cc:243] Waited 64.3 ms on lock 0x7b5400012320. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032b471 00007fb81033bdc0 00007fb8103acbd2 00007fb8103a22d7 00007fb8103a2114 00007fb810803ade 00007fb8108032b8 00007fb810807ba1 00007fb810807b59 00007fb810807ae9 00007fb810807ab1
0926 07:35:30.214696 (+ 1050us) log.cc:842] Serialized 356 byte log entry
0926 07:35:30.235978 (+ 21282us) op_driver.cc:444] REPLICATION: finished
0926 07:35:30.236544 (+ 566us) write_op.cc:301] APPLY: starting
0926 07:35:30.236620 (+ 76us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:30.236743 (+ 123us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:30.236779 (+ 36us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:30.237023 (+ 244us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:30.237241 (+ 218us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:30.237299 (+ 58us) write_op.cc:312] APPLY: finished
0926 07:35:30.237559 (+ 260us) log.cc:842] Serialized 23 byte log entry
0926 07:35:30.237722 (+ 163us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:30.237795 (+ 73us) write_op.cc:454] Released schema lock
0926 07:35:30.237841 (+ 46us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:30.238050 (+ 209us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:30.238082 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"spinlock_wait_cycles":19737344,"child_traces":[["op",{"apply.queue_time_us":201,"apply.run_cpu_time_us":1948,"apply.run_wall_time_us":1947,"num_ops":1,"prepare.queue_time_us":257831,"prepare.run_cpu_time_us":2093,"prepare.run_wall_time_us":98176,"raft.queue_time_us":78,"raft.run_cpu_time_us":564,"raft.run_wall_time_us":565,"replication_time_us":37581}],["op",{"apply.queue_time_us":368,"apply.run_cpu_time_us":1705,"apply.run_wall_time_us":1705,"num_ops":1,"prepare.queue_time_us":395326,"prepare.run_cpu_time_us":7249,"prepare.run_wall_time_us":123881,"raft.queue_time_us":81,"raft.run_cpu_time_us":733,"raft.run_wall_time_us":733,"replication_time_us":89742,"spinlock_wait_cycles":147986176}]]}
I20260926 07:35:30.830945 11754 tablet_bootstrap.cc:492] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:30.831358 11754 ts_tablet_manager.cc:1397] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.052s user 0.006s sys 0.003s
I20260926 07:35:30.807639 11871 rpcz_store.cc:276] 0926 07:35:29.136035 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.136223 (+ 188us) service_pool.cc:227] Handling call
0926 07:35:30.807113 (+1670890us) ts_tablet_manager.cc:1145] Deleting tablet 25b8b701563344bbbc986db2b0f39b9b
0926 07:35:30.807153 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:30.807350 (+ 197us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1670283}
W20260926 07:35:30.832906 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 25b8b701563344bbbc986db2b0f39b9b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25b8b701563344bbbc986db2b0f39b9b already in progress: creating tablet
I20260926 07:35:30.833777 11871 tablet_replica.cc:330] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:30.834537 11871 raft_consensus.cc:2229] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:30.834961 11871 raft_consensus.cc:2258] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:30.838442 12035 raft_consensus.cc:482] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:30.838857 12035 raft_consensus.cc:504] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.841024 12035 leader_election.cc:288] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:30.842782 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36c759963cb4915afdca7d12109cc6e" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
W20260926 07:35:30.844295 11249 leader_election.cc:341] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: must be running to vote when last-logged opid is not known
W20260926 07:35:30.848868 11104 consensus_peers.cc:487] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d -> Peer d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629): Couldn't send request to peer d6ef5b900d0f4e73a358f415efa97249. Status: Network error: Client connection negotiation failed: client connection to 127.10.171.62:37629: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260926 07:35:30.851692 11754 raft_consensus.cc:348] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.852355 11754 raft_consensus.cc:374] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:30.852567 11754 raft_consensus.cc:729] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:30.853829 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36c759963cb4915afdca7d12109cc6e" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:30.854866 11871 ts_tablet_manager.cc:1905] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 07:35:30.854928 11247 leader_election.cc:341] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:30.855279 11247 leader_election.cc:302] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade
I20260926 07:35:30.855647 11756 tablet_bootstrap.cc:654] T 034d508f0a164f3ea8308ea2d28818c4 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:30.855913 12035 raft_consensus.cc:2726] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:30.856578 11746 tablet_bootstrap.cc:654] T ae16af9edff9401589bcf54eeb3e8dfe P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:30.862450 11115 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_0cf0eeab44174032830f060f976b982e" } new_table_name: "default.Table_d6cdabd37ad94b339606911c37764fba" modify_external_catalogs: true
I20260926 07:35:30.864743 11120 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_53f51dc492ee41569d26cfe526418f7b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:30.866039 11120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_53f51dc492ee41569d26cfe526418f7b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:30.871897 11114 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_e0f192a8a56f4343afa3cb8b77b6148d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:30.873364 11114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e0f192a8a56f4343afa3cb8b77b6148d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:30.874377 11746 tablet_bootstrap.cc:492] T ae16af9edff9401589bcf54eeb3e8dfe P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:30.874723 11746 ts_tablet_manager.cc:1397] T ae16af9edff9401589bcf54eeb3e8dfe P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.062s user 0.008s sys 0.000s
I20260926 07:35:30.877053 11746 raft_consensus.cc:348] T ae16af9edff9401589bcf54eeb3e8dfe P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:30.877722 11746 raft_consensus.cc:374] T ae16af9edff9401589bcf54eeb3e8dfe P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:30.877439 11754 consensus_queue.cc:260] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.877959 11746 raft_consensus.cc:729] T ae16af9edff9401589bcf54eeb3e8dfe P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:30.882169 11746 consensus_queue.cc:260] T ae16af9edff9401589bcf54eeb3e8dfe P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:30.891534 11746 ts_tablet_manager.cc:1428] T ae16af9edff9401589bcf54eeb3e8dfe P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.017s user 0.007s sys 0.000s
I20260926 07:35:30.895315 11746 tablet_bootstrap.cc:492] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:30.904263 11754 ts_tablet_manager.cc:1428] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.073s user 0.007s sys 0.000s
I20260926 07:35:30.904938 11113 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 04180ace5d4548dfbcd226bf229d5c26
I20260926 07:35:30.905156 11746 tablet_bootstrap.cc:654] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:30.906143 11113 catalog_manager.cc:6565] ReplaceTablet: tablet 04180ace5d4548dfbcd226bf229d5c26 deleted and replaced by tablet b862f325a9df49079066dc62fe1a61f1
I20260926 07:35:30.908380 11582 tablet_service.cc:1514] Processing DeleteTablet for tablet 04180ace5d4548dfbcd226bf229d5c26 with delete_type TABLET_DATA_DELETED (replaced by tablet b862f325a9df49079066dc62fe1a61f1) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:30.910606 11440 tablet_service.cc:1514] Processing DeleteTablet for tablet 04180ace5d4548dfbcd226bf229d5c26 with delete_type TABLET_DATA_DELETED (replaced by tablet b862f325a9df49079066dc62fe1a61f1) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:30.910416 12108 tablet.cc:1722] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 07:35:30.912454 11754 tablet_bootstrap.cc:492] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:30.915879 11746 tablet_bootstrap.cc:492] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:30.916244 11746 ts_tablet_manager.cc:1397] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.000s
I20260926 07:35:30.915364 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_d0f2927c94ad48919b98e4f4199b42cd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:30.917840 11113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d0f2927c94ad48919b98e4f4199b42cd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:30.918497 11746 raft_consensus.cc:348] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.919061 11746 raft_consensus.cc:374] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:30.919281 11746 raft_consensus.cc:729] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:30.923849 11746 consensus_queue.cc:260] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.928661 11746 ts_tablet_manager.cc:1428] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260926 07:35:30.936426 11842 raft_consensus.cc:482] T b7837bc0821a4b68972b663606364955 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:30.936792 11842 raft_consensus.cc:504] T b7837bc0821a4b68972b663606364955 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:30.938975 11842 leader_election.cc:288] T b7837bc0821a4b68972b663606364955 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:30.940564 11746 tablet_bootstrap.cc:492] T 0218d2a727f942e48d770878e6bf4a54 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:30.942771 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7837bc0821a4b68972b663606364955" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:30.943495 11608 raft_consensus.cc:2379] T b7837bc0821a4b68972b663606364955 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 22f3902d01584936b20437606cb86dd1 in current term 1: Already voted for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in this term.
I20260926 07:35:30.946687 11752 ts_tablet_manager.cc:1918] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:30.947153 11752 log.cc:1196] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/640d21586bad4070957cff61bd174fd2
I20260926 07:35:30.948266 11752 ts_tablet_manager.cc:1939] T 640d21586bad4070957cff61bd174fd2 P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:30.949800 11754 tablet_bootstrap.cc:654] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:30.950131 11746 tablet_bootstrap.cc:654] T 0218d2a727f942e48d770878e6bf4a54 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:30.966908 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7837bc0821a4b68972b663606364955" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:30.969455 11286 tablet_service.cc:1514] Processing DeleteTablet for tablet 04180ace5d4548dfbcd226bf229d5c26 with delete_type TABLET_DATA_DELETED (replaced by tablet b862f325a9df49079066dc62fe1a61f1) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:30.970328 11465 raft_consensus.cc:3037] T ae16af9edff9401589bcf54eeb3e8dfe P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:30.974887 11401 leader_election.cc:302] T b7837bc0821a4b68972b663606364955 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 22f3902d01584936b20437606cb86dd1; no voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:30.975855 11842 raft_consensus.cc:3037] T b7837bc0821a4b68972b663606364955 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:30.982669 12035 consensus_queue.cc:1035] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:30.990677 11842 raft_consensus.cc:2726] T b7837bc0821a4b68972b663606364955 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:35:30.997252 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 3e28ca159d8349748812825ac1947c51 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3e28ca159d8349748812825ac1947c51 already in progress: creating tablet
W20260926 07:35:30.998689 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 21919d22b1d24546911d36384b01d9cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 21919d22b1d24546911d36384b01d9cc already in progress: creating tablet
I20260926 07:35:30.999663 11746 tablet_bootstrap.cc:492] T 0218d2a727f942e48d770878e6bf4a54 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:31.000875 12035 consensus_queue.cc:1035] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:31.006464 11754 tablet_bootstrap.cc:492] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:31.006870 11754 ts_tablet_manager.cc:1397] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.095s user 0.008s sys 0.000s
I20260926 07:35:31.013125 11871 ts_tablet_manager.cc:1918] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:31.013722 11871 log.cc:1196] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/c81a59361a6e42b7962b3e027c4a087f
I20260926 07:35:31.014673 11871 ts_tablet_manager.cc:1939] T c81a59361a6e42b7962b3e027c4a087f P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:31.015380 11116 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_b413c386b9e346668b23e2ad025e2b7b" } new_table_name: "default.Table_72783eb75b984ed394613e1333b94227" modify_external_catalogs: true
W20260926 07:35:31.020102 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 04180ace5d4548dfbcd226bf229d5c26 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 04180ace5d4548dfbcd226bf229d5c26 already in progress: creating tablet
I20260926 07:35:31.000008 11746 ts_tablet_manager.cc:1397] T 0218d2a727f942e48d770878e6bf4a54 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.060s user 0.016s sys 0.003s
I20260926 07:35:31.030006 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 107) took 2015 ms. Trace:
I20260926 07:35:31.047791 11754 raft_consensus.cc:348] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.048499 11754 raft_consensus.cc:374] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:31.048738 11754 raft_consensus.cc:729] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:31.049280 11754 consensus_queue.cc:260] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.051824 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 112) took 1916 ms. Trace:
I20260926 07:35:31.052130 11871 rpcz_store.cc:276] 0926 07:35:29.135141 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.138920 (+ 3779us) service_pool.cc:227] Handling call
0926 07:35:30.833612 (+1694692us) ts_tablet_manager.cc:1145] Deleting tablet c81a59361a6e42b7962b3e027c4a087f
0926 07:35:30.833654 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:30.922901 (+ 89247us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:31.012918 (+ 90017us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:31.051771 (+ 38853us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1682914}
I20260926 07:35:31.053263 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet c81a59361a6e42b7962b3e027c4a087f (table default.Table_1b1901dc5bbb4288be18304dd4a9b113 [id=707c7ee5ced14cee8b8005cce80dbbb5]) successfully deleted
I20260926 07:35:31.053417 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 114) took 1917 ms. Trace:
I20260926 07:35:31.053660 11871 rpcz_store.cc:276] 0926 07:35:29.135716 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.139581 (+ 3865us) service_pool.cc:227] Handling call
0926 07:35:31.053105 (+1913524us) ts_tablet_manager.cc:1145] Deleting tablet 5cd482fb159d418291cb02bc9138b076
0926 07:35:31.053145 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:31.053372 (+ 227us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1902138}
I20260926 07:35:31.054567 11871 tablet_replica.cc:330] T 640d21586bad4070957cff61bd174fd2 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
W20260926 07:35:31.061054 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 5cd482fb159d418291cb02bc9138b076 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5cd482fb159d418291cb02bc9138b076 already in progress: creating tablet
I20260926 07:35:31.051612 12107 tablet.cc:1722] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 07:35:31.065894 11756 tablet_bootstrap.cc:492] T 034d508f0a164f3ea8308ea2d28818c4 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:31.069654 11871 raft_consensus.cc:2229] T 640d21586bad4070957cff61bd174fd2 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:31.086123 11756 ts_tablet_manager.cc:1397] T 034d508f0a164f3ea8308ea2d28818c4 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.405s user 0.008s sys 0.002s
I20260926 07:35:31.032439 11842 raft_consensus.cc:482] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:31.022992 11746 raft_consensus.cc:348] T 0218d2a727f942e48d770878e6bf4a54 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.091151 11746 raft_consensus.cc:374] T 0218d2a727f942e48d770878e6bf4a54 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:31.091436 11746 raft_consensus.cc:729] T 0218d2a727f942e48d770878e6bf4a54 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:31.091184 11842 raft_consensus.cc:504] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.091995 11746 consensus_queue.cc:260] T 0218d2a727f942e48d770878e6bf4a54 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.093169 11842 leader_election.cc:288] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:31.093762 11746 ts_tablet_manager.cc:1428] T 0218d2a727f942e48d770878e6bf4a54 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.073s user 0.004s sys 0.000s
I20260926 07:35:31.097638 11746 tablet_bootstrap.cc:492] T c36c759963cb4915afdca7d12109cc6e P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:31.104040 11746 tablet_bootstrap.cc:654] T c36c759963cb4915afdca7d12109cc6e P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:31.104759 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c81a59361a6e42b7962b3e027c4a087f" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:31.098531 11752 rpcz_store.cc:276] 0926 07:35:29.014283 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.025715 (+ 11432us) service_pool.cc:227] Handling call
0926 07:35:30.472816 (+1447101us) ts_tablet_manager.cc:1145] Deleting tablet 640d21586bad4070957cff61bd174fd2
0926 07:35:30.472866 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:30.671035 (+198169us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:30.696037 (+ 25002us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:31.029852 (+333815us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1446470}
I20260926 07:35:31.107434 11871 raft_consensus.cc:2258] T 640d21586bad4070957cff61bd174fd2 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 07:35:31.107925 11401 leader_election.cc:341] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:31.109066 11871 ts_tablet_manager.cc:1905] T 640d21586bad4070957cff61bd174fd2 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:31.109371 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet 640d21586bad4070957cff61bd174fd2 (table default.Table_cd53ff22f9444f688e341de983e3e145 [id=fa5001d12cb14306b0a5ab0f0c79bc48]) successfully deleted
I20260926 07:35:31.117584 11746 tablet_bootstrap.cc:492] T c36c759963cb4915afdca7d12109cc6e P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:31.117944 11746 ts_tablet_manager.cc:1397] T c36c759963cb4915afdca7d12109cc6e P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.000s
I20260926 07:35:31.123682 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 108) took 2051 ms. Trace:
W20260926 07:35:31.163995 11546 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.171.1:41829, user_credentials={real_user=slave}} blocked reactor thread for 98847.6us
I20260926 07:35:31.166457 11754 ts_tablet_manager.cc:1428] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.159s user 0.014s sys 0.003s
I20260926 07:35:31.168403 11754 tablet_bootstrap.cc:492] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:31.168344 12125 raft_consensus.cc:482] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:31.168805 12125 raft_consensus.cc:504] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.170646 12125 leader_election.cc:288] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:31.183583 11754 tablet_bootstrap.cc:654] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:31.185338 11871 ts_tablet_manager.cc:1918] T 640d21586bad4070957cff61bd174fd2 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:31.188447 11871 log.cc:1196] T 640d21586bad4070957cff61bd174fd2 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/640d21586bad4070957cff61bd174fd2
I20260926 07:35:31.189308 11871 ts_tablet_manager.cc:1939] T 640d21586bad4070957cff61bd174fd2 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:31.194229 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 113) took 2058 ms. Trace:
I20260926 07:35:31.195690 11871 rpcz_store.cc:276] 0926 07:35:29.135430 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.139978 (+ 4548us) service_pool.cc:227] Handling call
0926 07:35:31.054447 (+1914469us) ts_tablet_manager.cc:1145] Deleting tablet 640d21586bad4070957cff61bd174fd2
0926 07:35:31.054481 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:31.157967 (+103486us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:31.185138 (+ 27171us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:31.194179 (+ 9041us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":51622,"tablet-delete.queue_time_us":1902693}
I20260926 07:35:31.197937 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 640d21586bad4070957cff61bd174fd2 (table default.Table_cd53ff22f9444f688e341de983e3e145 [id=fa5001d12cb14306b0a5ab0f0c79bc48]) successfully deleted
I20260926 07:35:31.198488 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "034d508f0a164f3ea8308ea2d28818c4" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:31.199009 11871 tablet_replica.cc:330] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:31.202708 11754 tablet_bootstrap.cc:492] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:31.204577 11754 ts_tablet_manager.cc:1397] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.037s user 0.011s sys 0.000s
I20260926 07:35:31.203244 11871 raft_consensus.cc:2229] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:31.199115 12137 tablet.cc:1722] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 07:35:31.208386 12134 raft_consensus.cc:482] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:31.209204 12134 raft_consensus.cc:504] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.207487 11871 raft_consensus.cc:2258] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:31.211498 11871 ts_tablet_manager.cc:1905] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:31.211596 12107 tablet.cc:1722] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260926 07:35:31.212450 12134 leader_election.cc:288] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:31.215406 11120 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_43b629da39c6487a8632545467cd9032" } modify_external_catalogs: true
I20260926 07:35:31.215979 11120 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_43b629da39c6487a8632545467cd9032" } modify_external_catalogs: true
I20260926 07:35:31.217471 11977 consensus_queue.cc:1035] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:31.223831 11114 master_service.cc:946] ReplaceTablet: received request to replace tablet 04180ace5d4548dfbcd226bf229d5c26 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:31.224488 11316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4239f5d48512403bb711ece401f0756b" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
W20260926 07:35:31.228631 11546 leader_election.cc:341] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:31.124301 11746 raft_consensus.cc:348] T c36c759963cb4915afdca7d12109cc6e P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.236279 11746 raft_consensus.cc:374] T c36c759963cb4915afdca7d12109cc6e P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:31.236573 11746 raft_consensus.cc:729] T c36c759963cb4915afdca7d12109cc6e P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:31.237246 11746 consensus_queue.cc:260] T c36c759963cb4915afdca7d12109cc6e P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.236032 11752 rpcz_store.cc:276] 0926 07:35:29.071997 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.078536 (+ 6539us) service_pool.cc:227] Handling call
0926 07:35:31.123356 (+2044820us) ts_tablet_manager.cc:1145] Deleting tablet 5cd482fb159d418291cb02bc9138b076
0926 07:35:31.123399 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:31.123617 (+ 218us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2036963}
I20260926 07:35:31.238480 11756 raft_consensus.cc:348] T 034d508f0a164f3ea8308ea2d28818c4 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.162918 12111 raft_consensus.cc:482] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:31.239573 12111 raft_consensus.cc:504] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:31.213784 11754 raft_consensus.cc:348] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.241278 11754 raft_consensus.cc:374] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:31.241480 12111 leader_election.cc:288] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:31.239197 11746 ts_tablet_manager.cc:1428] T c36c759963cb4915afdca7d12109cc6e P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.121s user 0.006s sys 0.000s
I20260926 07:35:31.241530 11754 raft_consensus.cc:729] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:31.242910 11746 tablet_bootstrap.cc:492] T 7e52ec7d802a4e90bccef777f77a2a1b P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
W20260926 07:35:31.128677 11402 leader_election.cc:341] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Not found: Tablet not found: c81a59361a6e42b7962b3e027c4a087f
I20260926 07:35:31.243520 11754 consensus_queue.cc:260] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.244113 11402 leader_election.cc:302] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 22f3902d01584936b20437606cb86dd1; no voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e
W20260926 07:35:31.245118 11402 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.3:45127, user_credentials={real_user=slave}} blocked reactor thread for 116465us
I20260926 07:35:31.245440 11754 ts_tablet_manager.cc:1428] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.040s user 0.005s sys 0.000s
I20260926 07:35:31.246265 11754 tablet_bootstrap.cc:492] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:31.247174 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4239f5d48512403bb711ece401f0756b" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:31.249223 11465 raft_consensus.cc:3037] T a8b629a4aba045b697e33bbebecb05b7 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:31.249001 11610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55ad1d96e2314d1eb0a717a3e9ff31bc" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:31.249980 11610 raft_consensus.cc:2454] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:31.250821 11746 tablet_bootstrap.cc:654] T 7e52ec7d802a4e90bccef777f77a2a1b P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:31.252436 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "034d508f0a164f3ea8308ea2d28818c4" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
W20260926 07:35:31.253172 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 5cd482fb159d418291cb02bc9138b076 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5cd482fb159d418291cb02bc9138b076 already in progress: creating tablet
I20260926 07:35:31.254328 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 109) took 2129 ms. Trace:
I20260926 07:35:31.254578 11752 rpcz_store.cc:276] 0926 07:35:29.125004 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.307875 (+182871us) service_pool.cc:227] Handling call
0926 07:35:31.254046 (+1946171us) ts_tablet_manager.cc:1145] Deleting tablet 25b8b701563344bbbc986db2b0f39b9b
0926 07:35:31.254091 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:31.254284 (+ 193us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1945643}
W20260926 07:35:31.255398 11545 leader_election.cc:341] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:31.255888 11545 leader_election.cc:302] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
W20260926 07:35:31.256858 11104 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 30df9b4afb414c90af5f4fb0c4582f01 (table default.Table_43b629da39c6487a8632545467cd9032 [id=156c350dad7246448e3f55e59820e470], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 30df9b4afb414c90af5f4fb0c4582f01 consensus state has no leader
I20260926 07:35:31.257314 11756 raft_consensus.cc:374] T 034d508f0a164f3ea8308ea2d28818c4 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:31.257547 11756 raft_consensus.cc:729] T 034d508f0a164f3ea8308ea2d28818c4 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:31.258059 12125 raft_consensus.cc:2726] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:35:31.259016 11401 leader_election.cc:341] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Not found: Tablet not found: 55ad1d96e2314d1eb0a717a3e9ff31bc
I20260926 07:35:31.259684 11871 ts_tablet_manager.cc:1918] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:31.259976 11871 log.cc:1196] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/e06ed286144941269b9a0c0c60bff0ce
I20260926 07:35:31.260670 11871 ts_tablet_manager.cc:1939] T e06ed286144941269b9a0c0c60bff0ce P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:31.261701 12137 tablet.cc:1722] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260926 07:35:31.262820 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 117) took 1651 ms. Trace:
I20260926 07:35:31.268307 11871 rpcz_store.cc:276] 0926 07:35:29.611599 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.611719 (+ 120us) service_pool.cc:227] Handling call
0926 07:35:31.198876 (+1587157us) ts_tablet_manager.cc:1145] Deleting tablet e06ed286144941269b9a0c0c60bff0ce
0926 07:35:31.198913 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:31.247338 (+ 48425us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:31.259489 (+ 12151us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:31.262772 (+ 3283us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1586532}
I20260926 07:35:31.269362 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 110) took 1588 ms. Trace:
I20260926 07:35:31.270066 11871 tablet_replica.cc:330] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:31.274284 11754 tablet_bootstrap.cc:654] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:31.276446 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet e06ed286144941269b9a0c0c60bff0ce (table default.Table_22c5272f755c4a659ac71d7a5e1c3a22 [id=bad324bd4f1e469ab9435b7a34c83cc4]) successfully deleted
W20260926 07:35:31.279913 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 25b8b701563344bbbc986db2b0f39b9b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25b8b701563344bbbc986db2b0f39b9b already in progress: creating tablet
I20260926 07:35:31.262643 11756 consensus_queue.cc:260] T 034d508f0a164f3ea8308ea2d28818c4 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.291105 11315 raft_consensus.cc:2454] T 034d508f0a164f3ea8308ea2d28818c4 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 0.
I20260926 07:35:31.275484 12111 raft_consensus.cc:2726] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:31.279897 11402 leader_election.cc:302] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters: e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:31.310702 11752 rpcz_store.cc:276] 0926 07:35:29.680991 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.681104 (+ 113us) service_pool.cc:227] Handling call
0926 07:35:31.255281 (+1574177us) ts_tablet_manager.cc:1145] Deleting tablet 9feaf1ed8ff34928b716fd0b4e41cdde
0926 07:35:31.255315 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:31.269312 (+ 13997us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1573632}
I20260926 07:35:31.311797 11752 tablet_replica.cc:330] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:31.311657 12111 raft_consensus.cc:2781] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:31.312350 12111 raft_consensus.cc:482] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:31.312774 12111 raft_consensus.cc:3037] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:31.313563 11756 ts_tablet_manager.cc:1428] T 034d508f0a164f3ea8308ea2d28818c4 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.227s user 0.005s sys 0.000s
I20260926 07:35:31.316310 11871 raft_consensus.cc:2229] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:31.318252 12108 tablet.cc:1722] T 4826318c83524f41840e770f33bd3696 P 22f3902d01584936b20437606cb86dd1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260926 07:35:31.328024 11754 tablet_bootstrap.cc:492] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:31.328421 11754 ts_tablet_manager.cc:1397] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.082s user 0.009s sys 0.000s
W20260926 07:35:31.328735 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 9feaf1ed8ff34928b716fd0b4e41cdde because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9feaf1ed8ff34928b716fd0b4e41cdde already in progress: creating tablet
I20260926 07:35:31.329108 12093 consensus_queue.cc:1035] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 07:35:31.337962 11746 tablet_bootstrap.cc:492] T 7e52ec7d802a4e90bccef777f77a2a1b P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:31.338467 11746 ts_tablet_manager.cc:1397] T 7e52ec7d802a4e90bccef777f77a2a1b P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.096s user 0.009s sys 0.001s
I20260926 07:35:31.340807 11746 raft_consensus.cc:348] T 7e52ec7d802a4e90bccef777f77a2a1b P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.342872 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_f3f760b496a54e37b41668ee7a9585d5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:31.343797 12133 raft_consensus.cc:482] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260926 07:35:31.344302 11113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f3f760b496a54e37b41668ee7a9585d5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:31.344219 12133 raft_consensus.cc:504] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.346233 12133 leader_election.cc:288] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:31.347028 11871 raft_consensus.cc:2258] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:31.350659 11871 ts_tablet_manager.cc:1905] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:31.351132 11754 raft_consensus.cc:348] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.351787 11754 raft_consensus.cc:374] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:31.352033 11754 raft_consensus.cc:729] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:31.352566 11754 consensus_queue.cc:260] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:31.352909 11545 leader_election.cc:341] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:31.357419 11546 leader_election.cc:302] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1
W20260926 07:35:31.358475 11546 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.1:41829, user_credentials={real_user=slave}} blocked reactor thread for 64144.9us
I20260926 07:35:31.358892 12125 raft_consensus.cc:2781] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:31.359208 12125 raft_consensus.cc:482] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:31.359499 12125 raft_consensus.cc:3037] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:31.360267 11316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e52ec7d802a4e90bccef777f77a2a1b" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
W20260926 07:35:31.361866 11546 leader_election.cc:341] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:31.378933 12147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.171.62:37629 every 8 connection(s)
I20260926 07:35:31.381696 11756 tablet_bootstrap.cc:492] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:31.379531 11716 rpc_server.cc:307] RPC server started. Bound to: 127.10.171.62:37629
I20260926 07:35:31.397851 11756 tablet_bootstrap.cc:654] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:31.398447 11746 raft_consensus.cc:374] T 7e52ec7d802a4e90bccef777f77a2a1b P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:31.398717 11746 raft_consensus.cc:729] T 7e52ec7d802a4e90bccef777f77a2a1b P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:31.335667 12111 raft_consensus.cc:504] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:31.401535 12111 leader_election.cc:288] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:31.387240 11752 raft_consensus.cc:2229] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:31.402403 11752 raft_consensus.cc:2258] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:31.402663 11716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/master-0/data/info.pb
I20260926 07:35:31.403019 11610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55ad1d96e2314d1eb0a717a3e9ff31bc" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:31.403565 11610 raft_consensus.cc:3037] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:31.404313 11752 ts_tablet_manager.cc:1905] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:31.404788 10924 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu as pid 11716
I20260926 07:35:31.410848 11610 raft_consensus.cc:2454] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
W20260926 07:35:31.411605 11401 leader_election.cc:341] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Not found: Tablet not found: 55ad1d96e2314d1eb0a717a3e9ff31bc
I20260926 07:35:31.412542 11402 leader_election.cc:302] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters: e9bfebb34fa14cd2ad4843fffbe5c54e
W20260926 07:35:31.414199 11545 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.2:42379, user_credentials={real_user=slave}} blocked reactor thread for 133534us
I20260926 07:35:31.416404 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e52ec7d802a4e90bccef777f77a2a1b" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:31.417896 12133 raft_consensus.cc:482] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:31.418335 12133 raft_consensus.cc:504] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.420073 12133 leader_election.cc:288] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:31.421082 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04180ace5d4548dfbcd226bf229d5c26" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:31.422096 11754 ts_tablet_manager.cc:1428] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.093s user 0.012s sys 0.003s
I20260926 07:35:31.422986 11754 tablet_bootstrap.cc:492] T 90f62593b6f844fc8a41b9be240101fd P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
W20260926 07:35:31.422938 11546 leader_election.cc:341] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:31.427369 12111 raft_consensus.cc:2781] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:31.428877 12111 raft_consensus.cc:686] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:31.429615 12111 consensus_queue.cc:237] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:31.429952 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04180ace5d4548dfbcd226bf229d5c26" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
W20260926 07:35:31.431993 11545 leader_election.cc:341] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:31.433122 11754 tablet_bootstrap.cc:654] T 90f62593b6f844fc8a41b9be240101fd P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:31.432516 11545 leader_election.cc:302] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:31.436095 12133 raft_consensus.cc:2726] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:31.438275 11842 raft_consensus.cc:482] T 0218d2a727f942e48d770878e6bf4a54 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:31.438681 11842 raft_consensus.cc:504] T 0218d2a727f942e48d770878e6bf4a54 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.441143 11842 leader_election.cc:288] T 0218d2a727f942e48d770878e6bf4a54 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:31.443331 11610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0218d2a727f942e48d770878e6bf4a54" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:31.443534 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0218d2a727f942e48d770878e6bf4a54" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:31.444267 11754 tablet_bootstrap.cc:492] T 90f62593b6f844fc8a41b9be240101fd P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:31.444651 11754 ts_tablet_manager.cc:1397] T 90f62593b6f844fc8a41b9be240101fd P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.022s user 0.008s sys 0.001s
W20260926 07:35:31.444710 11401 leader_election.cc:341] T 0218d2a727f942e48d770878e6bf4a54 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
W20260926 07:35:31.445618 11402 leader_election.cc:341] T 0218d2a727f942e48d770878e6bf4a54 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:31.445966 11402 leader_election.cc:302] T 0218d2a727f942e48d770878e6bf4a54 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 22f3902d01584936b20437606cb86dd1; no voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:31.446798 11842 raft_consensus.cc:2726] T 0218d2a727f942e48d770878e6bf4a54 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:31.446813 11754 raft_consensus.cc:348] T 90f62593b6f844fc8a41b9be240101fd P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.447420 11754 raft_consensus.cc:374] T 90f62593b6f844fc8a41b9be240101fd P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:31.447685 11754 raft_consensus.cc:729] T 90f62593b6f844fc8a41b9be240101fd P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:31.448357 11113 master_service.cc:946] ReplaceTablet: received request to replace tablet 034d508f0a164f3ea8308ea2d28818c4 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:31.399262 11746 consensus_queue.cc:260] T 7e52ec7d802a4e90bccef777f77a2a1b P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.452736 11463 raft_consensus.cc:2454] T 7e52ec7d802a4e90bccef777f77a2a1b P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 0.
I20260926 07:35:31.454221 11545 leader_election.cc:302] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters: e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:31.455025 12133 raft_consensus.cc:2781] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:31.455538 11746 ts_tablet_manager.cc:1428] T 7e52ec7d802a4e90bccef777f77a2a1b P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.117s user 0.004s sys 0.000s
I20260926 07:35:31.455765 12133 raft_consensus.cc:482] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:31.456071 12133 raft_consensus.cc:3037] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:31.456372 11746 tablet_bootstrap.cc:492] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:31.456394 11754 consensus_queue.cc:260] T 90f62593b6f844fc8a41b9be240101fd P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.458357 11754 ts_tablet_manager.cc:1428] T 90f62593b6f844fc8a41b9be240101fd P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.013s user 0.004s sys 0.000s
I20260926 07:35:31.459163 11754 tablet_bootstrap.cc:492] T 0218d2a727f942e48d770878e6bf4a54 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:31.460815 12125 raft_consensus.cc:504] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.463824 12125 leader_election.cc:288] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:31.463675 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "034d508f0a164f3ea8308ea2d28818c4" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:31.464272 11313 raft_consensus.cc:3037] T 034d508f0a164f3ea8308ea2d28818c4 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:31.477751 11313 raft_consensus.cc:2454] T 034d508f0a164f3ea8308ea2d28818c4 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 1.
I20260926 07:35:31.478929 11546 leader_election.cc:302] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:31.479799 12125 raft_consensus.cc:2781] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:31.462867 12133 raft_consensus.cc:504] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.482925 12133 leader_election.cc:288] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:31.486280 12125 raft_consensus.cc:686] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Becoming Leader. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Running, Role: LEADER
I20260926 07:35:31.490418 11842 raft_consensus.cc:482] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:31.491313 11842 raft_consensus.cc:504] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:31.487438 12125 consensus_queue.cc:237] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.494207 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e52ec7d802a4e90bccef777f77a2a1b" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:31.494773 11465 raft_consensus.cc:3037] T 7e52ec7d802a4e90bccef777f77a2a1b P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:31.489353 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e52ec7d802a4e90bccef777f77a2a1b" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
W20260926 07:35:31.496515 11546 leader_election.cc:341] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
W20260926 07:35:31.502226 11402 leader_election.cc:341] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Not found: Tablet not found: d6fc6178f1e54c3984a20873864233cd
I20260926 07:35:31.504307 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "034d508f0a164f3ea8308ea2d28818c4" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:31.504935 12093 consensus_queue.cc:1035] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:31.505910 11842 leader_election.cc:288] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:31.503371 11871 ts_tablet_manager.cc:1918] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:31.507642 11871 log.cc:1196] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/509f607b089c44bc9c3f80ed59a68c95
I20260926 07:35:31.509536 11754 tablet_bootstrap.cc:654] T 0218d2a727f942e48d770878e6bf4a54 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:31.510845 11465 raft_consensus.cc:2454] T 7e52ec7d802a4e90bccef777f77a2a1b P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 1.
I20260926 07:35:31.511432 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fc6178f1e54c3984a20873864233cd" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:31.511999 11313 raft_consensus.cc:2454] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:31.514379 11401 leader_election.cc:302] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 6070e9e3d1234a9d83e49431416afade
I20260926 07:35:31.515491 11842 raft_consensus.cc:2781] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:31.515875 11842 raft_consensus.cc:482] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:31.516335 11842 raft_consensus.cc:3037] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:31.520663 12134 raft_consensus.cc:482] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:31.521078 12134 raft_consensus.cc:504] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.523295 12134 leader_election.cc:288] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:31.525183 11871 ts_tablet_manager.cc:1939] T 509f607b089c44bc9c3f80ed59a68c95 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:31.527257 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36c759963cb4915afdca7d12109cc6e" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:31.527829 11313 raft_consensus.cc:2454] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 0.
I20260926 07:35:31.529213 11546 leader_election.cc:302] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:31.532802 12133 raft_consensus.cc:2781] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:31.533123 12133 raft_consensus.cc:482] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:31.533419 12133 raft_consensus.cc:3037] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:31.533960 12134 raft_consensus.cc:482] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:31.534379 12134 raft_consensus.cc:504] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.536899 12134 leader_election.cc:288] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:31.537572 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36c759963cb4915afdca7d12109cc6e" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:31.538208 11465 raft_consensus.cc:2454] T c36c759963cb4915afdca7d12109cc6e P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 0.
I20260926 07:35:31.540151 11756 tablet_bootstrap.cc:492] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:31.540594 11756 ts_tablet_manager.cc:1397] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.159s user 0.008s sys 0.003s
I20260926 07:35:31.538811 11752 ts_tablet_manager.cc:1918] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 07:35:31.541631 11752 log.cc:1196] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/a000cb9928de4459be1e6b1b7a81848a
I20260926 07:35:31.542387 11752 ts_tablet_manager.cc:1939] T a000cb9928de4459be1e6b1b7a81848a P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:31.545184 11842 raft_consensus.cc:504] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:31.548004 11746 tablet_bootstrap.cc:654] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:31.551251 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04be706330924282961054c5835cec73" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:31.552029 11313 raft_consensus.cc:2454] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 0.
I20260926 07:35:31.553803 12154 sys_catalog.cc:263] Verifying existing consensus state
I20260926 07:35:31.559137 11756 raft_consensus.cc:348] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.573223 11546 leader_election.cc:302] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:31.573939 12125 raft_consensus.cc:2781] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:31.574239 12125 raft_consensus.cc:482] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:31.574492 12125 raft_consensus.cc:3037] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
W20260926 07:35:31.574918 11402 leader_election.cc:341] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Not found: Tablet not found: d6fc6178f1e54c3984a20873864233cd
W20260926 07:35:31.578758 11545 leader_election.cc:341] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:31.579634 11842 leader_election.cc:288] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:31.580314 11545 leader_election.cc:302] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters: e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:31.581005 12134 raft_consensus.cc:2781] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election won for term 1
W20260926 07:35:31.581358 11545 consensus_peers.cc:487] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade -> Peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Couldn't send request to peer 22f3902d01584936b20437606cb86dd1. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:31.581398 12134 raft_consensus.cc:686] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Becoming Leader. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Running, Role: LEADER
I20260926 07:35:31.582389 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fc6178f1e54c3984a20873864233cd" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:31.582180 12134 consensus_queue.cc:237] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.582912 11313 raft_consensus.cc:3037] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:31.583079 12125 raft_consensus.cc:504] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.585242 12069 consensus_queue.cc:1035] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 07:35:31.580534 11756 raft_consensus.cc:374] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:31.587098 11963 raft_consensus.cc:482] T 90f62593b6f844fc8a41b9be240101fd P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:31.587630 11963 raft_consensus.cc:504] T 90f62593b6f844fc8a41b9be240101fd P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.589751 11963 leader_election.cc:288] T 90f62593b6f844fc8a41b9be240101fd P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:31.586700 11756 raft_consensus.cc:729] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:31.591414 11316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90f62593b6f844fc8a41b9be240101fd" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
W20260926 07:35:31.593345 11546 leader_election.cc:341] T 90f62593b6f844fc8a41b9be240101fd P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:31.594338 11756 consensus_queue.cc:260] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.596262 11756 ts_tablet_manager.cc:1428] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.055s user 0.006s sys 0.000s
I20260926 07:35:31.597012 11756 tablet_bootstrap.cc:492] T 3e28ca159d8349748812825ac1947c51 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:31.607990 11120 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 30df9b4afb414c90af5f4fb0c4582f01
I20260926 07:35:31.610509 11292 tablet_service.cc:1514] Processing DeleteTablet for tablet 30df9b4afb414c90af5f4fb0c4582f01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:31 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:31.610833 11440 tablet_service.cc:1514] Processing DeleteTablet for tablet 30df9b4afb414c90af5f4fb0c4582f01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:31 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:31.611941 11120 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_19ac6fcbe1b34c2d9f649f43ab3bdffd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:31.614050 11120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_19ac6fcbe1b34c2d9f649f43ab3bdffd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:31.619284 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90f62593b6f844fc8a41b9be240101fd" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:31.620715 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04be706330924282961054c5835cec73" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:31.622799 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 111) took 1789 ms. Trace:
I20260926 07:35:31.623142 11752 rpcz_store.cc:276] 0926 07:35:29.833325 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.833452 (+ 127us) service_pool.cc:227] Handling call
0926 07:35:31.311647 (+1478195us) ts_tablet_manager.cc:1145] Deleting tablet a000cb9928de4459be1e6b1b7a81848a
0926 07:35:31.311689 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:31.495942 (+184253us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:31.531610 (+ 35668us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:31.622737 (+ 91127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1477635}
I20260926 07:35:31.624370 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet a000cb9928de4459be1e6b1b7a81848a (table default.Table_d0380e9d3f26439bbc91cf476aeae69f [id=19875953f2b6484f934503404af57593]) successfully deleted
I20260926 07:35:31.624631 11582 tablet_service.cc:1514] Processing DeleteTablet for tablet 30df9b4afb414c90af5f4fb0c4582f01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:31 UTC) from {username='slave'} at 127.0.0.1:35244
W20260926 07:35:31.625407 11545 leader_election.cc:341] T 90f62593b6f844fc8a41b9be240101fd P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:31.626011 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 118) took 1974 ms. Trace:
I20260926 07:35:31.626256 11746 tablet_bootstrap.cc:492] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:31.626724 11746 ts_tablet_manager.cc:1397] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.170s user 0.011s sys 0.000s
I20260926 07:35:31.629061 11111 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet a0bdf6a4656e4217b0a7013fddc4cca8 (table default.Table_1da003049e4b46eb994aaa3f3094d63c [id=3ce63f9fb5ad4b82ba2fbbf2e3678271]). Expected version 2 got 1
I20260926 07:35:31.629359 11313 raft_consensus.cc:2454] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:31.629103 11746 raft_consensus.cc:348] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.629846 11746 raft_consensus.cc:374] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:31.630364 11111 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 4826318c83524f41840e770f33bd3696 (table default.Table_0cf0eeab44174032830f060f976b982e [id=55a6e4b7598043d8bf49512f2542c589]). Expected version 3 got 0
I20260926 07:35:31.630456 11401 leader_election.cc:302] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 6070e9e3d1234a9d83e49431416afade
I20260926 07:35:31.631295 12125 leader_election.cc:288] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:31.625869 11545 leader_election.cc:302] T 90f62593b6f844fc8a41b9be240101fd P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:31.632686 12164 raft_consensus.cc:2781] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:31.632651 11316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04be706330924282961054c5835cec73" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:31.628712 11871 rpcz_store.cc:276] 0926 07:35:29.651631 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.651747 (+ 116us) service_pool.cc:227] Handling call
0926 07:35:31.269934 (+1618187us) ts_tablet_manager.cc:1145] Deleting tablet 509f607b089c44bc9c3f80ed59a68c95
0926 07:35:31.269968 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:31.449154 (+179186us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:31.503099 (+ 53945us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:31.625940 (+122841us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":26682,"tablet-delete.queue_time_us":1617623}
I20260926 07:35:31.633270 11316 raft_consensus.cc:3037] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:31.633697 11871 tablet_replica.cc:330] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:31.634125 11756 tablet_bootstrap.cc:654] T 3e28ca159d8349748812825ac1947c51 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:31.634460 11871 raft_consensus.cc:2229] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:31.634909 11871 raft_consensus.cc:2258] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:31.630108 11746 raft_consensus.cc:729] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
W20260926 07:35:31.636651 11545 leader_election.cc:341] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:31.637827 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 509f607b089c44bc9c3f80ed59a68c95 (table default.Table_65217d931d594a70b45c7d5c486930ba [id=b0949f4718f8482881854a65f89d7421]) successfully deleted
I20260926 07:35:31.637385 11746 consensus_queue.cc:260] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.629822 12133 raft_consensus.cc:504] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.638644 12164 raft_consensus.cc:686] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:31.640034 12133 leader_election.cc:288] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:31.640148 11118 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet a0bdf6a4656e4217b0a7013fddc4cca8 (table default.Table_1da003049e4b46eb994aaa3f3094d63c [id=3ce63f9fb5ad4b82ba2fbbf2e3678271]). Expected version 2 got 1
I20260926 07:35:31.640527 12133 raft_consensus.cc:2726] T 90f62593b6f844fc8a41b9be240101fd P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:31.640894 11118 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 4826318c83524f41840e770f33bd3696 (table default.Table_0cf0eeab44174032830f060f976b982e [id=55a6e4b7598043d8bf49512f2542c589]). Expected version 3 got 0
I20260926 07:35:31.628291 11754 tablet_bootstrap.cc:492] T 0218d2a727f942e48d770878e6bf4a54 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:31.640921 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36c759963cb4915afdca7d12109cc6e" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:31.641500 11754 ts_tablet_manager.cc:1397] T 0218d2a727f942e48d770878e6bf4a54 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.182s user 0.011s sys 0.004s
I20260926 07:35:31.641486 11118 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet ae16af9edff9401589bcf54eeb3e8dfe (table default.Table_f9178d5e3d934f1f9ef267a782316458 [id=0c18461963f444269787b667b3648709]). Expected version 2 got 0
I20260926 07:35:31.641559 11315 raft_consensus.cc:3037] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:31.642773 11118 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 0eae53ce278749208492815b918582f3 (table default.Table_56493b3efdaa43e684afcd3d4b5dffdd [id=f66bffa952604b28bb51299de795ee2a]). Expected version 1 got 0
I20260926 07:35:31.643217 11316 raft_consensus.cc:2454] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 1.
I20260926 07:35:31.643317 11118 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 034d508f0a164f3ea8308ea2d28818c4 (table default.Table_53678f9c4c01403eb2d846d3242be0aa [id=db19d49c9c29488e89f7280e91a11713]). Expected version 1 got 0
W20260926 07:35:31.645366 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 3e28ca159d8349748812825ac1947c51 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3e28ca159d8349748812825ac1947c51 already in progress: creating tablet
I20260926 07:35:31.648921 11286 tablet_service.cc:1467] Processing CreateTablet for tablet 0b1b14a340a7474fa8e76a464b0838e1 (DEFAULT_TABLE table=default.Table_e64de6cf413946c79498bb477ac3f3ac [id=f476e94c06e846a5bc846f2760a9979a]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 07:35:31.649406 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 30df9b4afb414c90af5f4fb0c4582f01 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 30df9b4afb414c90af5f4fb0c4582f01 already in progress: creating tablet
I20260926 07:35:31.650017 11286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b1b14a340a7474fa8e76a464b0838e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:31.654830 11546 leader_election.cc:302] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:31.656572 12133 raft_consensus.cc:2781] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:31.656996 12133 raft_consensus.cc:686] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Becoming Leader. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Running, Role: LEADER
I20260926 07:35:31.657711 12133 consensus_queue.cc:237] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.643805 11754 raft_consensus.cc:348] T 0218d2a727f942e48d770878e6bf4a54 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.659905 11754 raft_consensus.cc:374] T 0218d2a727f942e48d770878e6bf4a54 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:31.660183 11754 raft_consensus.cc:729] T 0218d2a727f942e48d770878e6bf4a54 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:31.660746 11754 consensus_queue.cc:260] T 0218d2a727f942e48d770878e6bf4a54 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.665823 11754 ts_tablet_manager.cc:1428] T 0218d2a727f942e48d770878e6bf4a54 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.024s user 0.003s sys 0.003s
I20260926 07:35:31.668080 11754 tablet_bootstrap.cc:492] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:31.668145 11660 heartbeater.cc:347] Connected to a master server at 127.10.171.62:37629
I20260926 07:35:31.668641 11660 heartbeater.cc:503] Master 127.10.171.62:37629 was elected leader, sending a full tablet report...
I20260926 07:35:31.672781 11370 heartbeater.cc:347] Connected to a master server at 127.10.171.62:37629
I20260926 07:35:31.678316 11754 tablet_bootstrap.cc:654] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:31.680423 11315 raft_consensus.cc:2454] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 1.
I20260926 07:35:31.640491 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04be706330924282961054c5835cec73" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:31.682197 11546 leader_election.cc:302] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:31.682937 12125 raft_consensus.cc:2781] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:31.683339 12125 raft_consensus.cc:686] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Becoming Leader. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Running, Role: LEADER
W20260926 07:35:31.683362 11545 leader_election.cc:341] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:31.684059 12125 consensus_queue.cc:237] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.689852 11746 ts_tablet_manager.cc:1428] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.063s user 0.008s sys 0.000s
I20260926 07:35:31.690807 11746 tablet_bootstrap.cc:492] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:31.646436 11752 tablet_replica.cc:330] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:31.649812 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36c759963cb4915afdca7d12109cc6e" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:31.655876 11842 raft_consensus.cc:482] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:31.680976 11515 heartbeater.cc:347] Connected to a master server at 127.10.171.62:37629
I20260926 07:35:31.694229 11842 raft_consensus.cc:504] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.696031 11842 leader_election.cc:288] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:31.697377 11610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "034d508f0a164f3ea8308ea2d28818c4" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:31.697947 11463 raft_consensus.cc:3037] T c36c759963cb4915afdca7d12109cc6e P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:31.682571 12164 consensus_queue.cc:237] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:31.706179 11463 raft_consensus.cc:2454] T c36c759963cb4915afdca7d12109cc6e P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 1.
I20260926 07:35:31.707852 11752 raft_consensus.cc:2229] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:31.708349 11752 raft_consensus.cc:2258] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:31.711341 11120 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_11baf2ea5fdc431fae1ae6d0dd3cdb2b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:31.716712 11756 tablet_bootstrap.cc:492] T 3e28ca159d8349748812825ac1947c51 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:31.719679 11756 ts_tablet_manager.cc:1397] T 3e28ca159d8349748812825ac1947c51 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.123s user 0.010s sys 0.000s
I20260926 07:35:31.721035 11440 tablet_service.cc:1467] Processing CreateTablet for tablet 0b1b14a340a7474fa8e76a464b0838e1 (DEFAULT_TABLE table=default.Table_e64de6cf413946c79498bb477ac3f3ac [id=f476e94c06e846a5bc846f2760a9979a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:31.722069 11440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b1b14a340a7474fa8e76a464b0838e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:31.730610 11746 tablet_bootstrap.cc:654] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:31.732654 11756 raft_consensus.cc:348] T 3e28ca159d8349748812825ac1947c51 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.734207 11756 raft_consensus.cc:374] T 3e28ca159d8349748812825ac1947c51 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:31.734462 11756 raft_consensus.cc:729] T 3e28ca159d8349748812825ac1947c51 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:31.734941 11754 tablet_bootstrap.cc:492] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:31.735431 11754 ts_tablet_manager.cc:1397] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.068s user 0.007s sys 0.018s
I20260926 07:35:31.737422 11754 raft_consensus.cc:348] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.738020 11754 raft_consensus.cc:374] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:31.738247 11754 raft_consensus.cc:729] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:31.738303 11582 tablet_service.cc:1467] Processing CreateTablet for tablet 0b1b14a340a7474fa8e76a464b0838e1 (DEFAULT_TABLE table=default.Table_e64de6cf413946c79498bb477ac3f3ac [id=f476e94c06e846a5bc846f2760a9979a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:31.723712 11113 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 034d508f0a164f3ea8308ea2d28818c4
I20260926 07:35:31.738858 11754 consensus_queue.cc:260] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.739341 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "034d508f0a164f3ea8308ea2d28818c4" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:31.740527 11445 tablet_service.cc:1514] Processing DeleteTablet for tablet 034d508f0a164f3ea8308ea2d28818c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 4088ccb81e844cc29e1735bb341e4a92) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:31.741078 11313 raft_consensus.cc:2379] T 034d508f0a164f3ea8308ea2d28818c4 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 22f3902d01584936b20437606cb86dd1 in current term 1: Already voted for candidate 6070e9e3d1234a9d83e49431416afade in this term.
I20260926 07:35:31.741153 11286 tablet_service.cc:1514] Processing DeleteTablet for tablet 034d508f0a164f3ea8308ea2d28818c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 4088ccb81e844cc29e1735bb341e4a92) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:31.741475 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet 034d508f0a164f3ea8308ea2d28818c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 4088ccb81e844cc29e1735bb341e4a92) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:31.741955 11113 catalog_manager.cc:6565] ReplaceTablet: tablet 034d508f0a164f3ea8308ea2d28818c4 deleted and replaced by tablet 4088ccb81e844cc29e1735bb341e4a92
I20260926 07:35:31.742002 12168 tablet_replica.cc:330] T 034d508f0a164f3ea8308ea2d28818c4 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:31.741837 11756 consensus_queue.cc:260] T 3e28ca159d8349748812825ac1947c51 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.742192 11401 leader_election.cc:302] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 22f3902d01584936b20437606cb86dd1; no voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:31.742830 12168 raft_consensus.cc:2229] T 034d508f0a164f3ea8308ea2d28818c4 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:31.743211 11754 ts_tablet_manager.cc:1428] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20260926 07:35:31.743273 11842 raft_consensus.cc:3037] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:31.743643 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_a2d61651f47c4fd599f740faeec0f7d3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:31.745064 11113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a2d61651f47c4fd599f740faeec0f7d3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:31.749343 12168 raft_consensus.cc:2258] T 034d508f0a164f3ea8308ea2d28818c4 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:31.749670 11842 raft_consensus.cc:2726] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:31.730795 11114 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet ae16af9edff9401589bcf54eeb3e8dfe (table default.Table_f9178d5e3d934f1f9ef267a782316458 [id=0c18461963f444269787b667b3648709]). Expected version 2 got 0
I20260926 07:35:31.751060 12168 ts_tablet_manager.cc:1905] T 034d508f0a164f3ea8308ea2d28818c4 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:31.751523 11114 catalog_manager.cc:5495] T b7837bc0821a4b68972b663606364955 P e9bfebb34fa14cd2ad4843fffbe5c54e reported cstate change: term changed from 0 to 1, leader changed from <none> to e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1). New cstate: current_term: 1 leader_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } health_report { overall_health: UNKNOWN } } }
I20260926 07:35:31.753939 11115 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:44836 (request call id 213) took 1019 ms. Trace:
I20260926 07:35:31.756467 11752 ts_tablet_manager.cc:1905] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:31.754249 11115 rpcz_store.cc:276] 0926 07:35:30.734346 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:30.776769 (+ 42423us) service_pool.cc:227] Handling call
0926 07:35:30.861815 (+ 85046us) spinlock_profiling.cc:243] Waited 82.2 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb8123767d6 00007fb80febc634 00007fb80febc5c2 00007fb80febc4f1 00007fb80febc46c 00007fb80febb722 00007fb80e15594c 00007fb80e154d86
0926 07:35:30.863098 (+ 1283us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0926 07:35:30.863819 (+ 721us) catalog_manager.cc:3616] Apply alter schema
0926 07:35:30.868042 (+ 4223us) catalog_manager.cc:3639] Acquired catalog manager lock
0926 07:35:30.868278 (+ 236us) catalog_manager.cc:3786] Updating metadata on disk
0926 07:35:31.210976 (+342698us) catalog_manager.cc:3817] Committing alterations to in-memory state
0926 07:35:31.753887 (+542911us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:30.940008 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:30.940418 (+ 410us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:30.940446 (+ 28us) write_op.cc:435] Acquired schema lock
0926 07:35:30.940463 (+ 17us) tablet.cc:660] Decoding operations
0926 07:35:31.010901 (+ 70438us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:31.010961 (+ 60us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:31.010983 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0926 07:35:31.011134 (+ 151us) tablet.cc:699] Row locks acquired
0926 07:35:31.011147 (+ 13us) write_op.cc:260] PREPARE: finished
0926 07:35:31.011249 (+ 102us) write_op.cc:270] Start()
0926 07:35:31.011322 (+ 73us) write_op.cc:276] Timestamp: P: 1790408131011171 usec, L: 0
0926 07:35:31.011337 (+ 15us) op_driver.cc:331] REPLICATION: starting
0926 07:35:31.011768 (+ 431us) log.cc:842] Serialized 405 byte log entry
0926 07:35:31.192645 (+180877us) op_driver.cc:444] REPLICATION: finished
0926 07:35:31.207948 (+ 15303us) write_op.cc:301] APPLY: starting
0926 07:35:31.208000 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:31.208095 (+ 95us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:31.208232 (+ 137us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:31.208375 (+ 143us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:31.208497 (+ 122us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:31.208541 (+ 44us) write_op.cc:312] APPLY: finished
0926 07:35:31.208704 (+ 163us) log.cc:842] Serialized 23 byte log entry
0926 07:35:31.208857 (+ 153us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:31.208912 (+ 55us) write_op.cc:454] Released schema lock
0926 07:35:31.208939 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:31.209113 (+ 174us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:31.209136 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":189009280,"child_traces":[["op",{"apply.queue_time_us":15177,"apply.run_cpu_time_us":1322,"apply.run_wall_time_us":1328,"num_ops":1,"prepare.queue_time_us":70733,"prepare.run_cpu_time_us":1916,"prepare.run_wall_time_us":148213,"raft.queue_time_us":56273,"raft.run_cpu_time_us":15,"raft.run_wall_time_us":16,"replication_time_us":181235}]]}
I20260926 07:35:31.760161 11115 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_adafc8d10ea04a3ea42e2766739e9451"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:31.762333 11115 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_adafc8d10ea04a3ea42e2766739e9451 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:31.771173 12108 tablet.cc:1722] T 4826318c83524f41840e770f33bd3696 P 22f3902d01584936b20437606cb86dd1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260926 07:35:31.773260 11756 ts_tablet_manager.cc:1428] T 3e28ca159d8349748812825ac1947c51 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.053s user 0.006s sys 0.000s
I20260926 07:35:31.781260 11286 tablet_service.cc:1467] Processing CreateTablet for tablet f7459bf97b7f4b648d8ca6218cd72914 (DEFAULT_TABLE table=default.Table_170efaa4a249400e8c1d772a5ac0ec54 [id=e3d379a69ef94c6a990d3dfbea076591]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 07:35:31.768576 11120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11baf2ea5fdc431fae1ae6d0dd3cdb2b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:31.787699 12168 ts_tablet_manager.cc:1918] T 034d508f0a164f3ea8308ea2d28818c4 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:31.790220 11871 ts_tablet_manager.cc:1905] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:31.791771 11582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b1b14a340a7474fa8e76a464b0838e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:31.794204 11754 tablet_bootstrap.cc:492] T 3f581510fe624127a4d27d302f5be614 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:31.803112 11754 tablet_bootstrap.cc:654] T 3f581510fe624127a4d27d302f5be614 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:31.821758 11752 ts_tablet_manager.cc:1918] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:31.822098 11752 log.cc:1196] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/509f607b089c44bc9c3f80ed59a68c95
I20260926 07:35:31.822795 11752 ts_tablet_manager.cc:1939] T 509f607b089c44bc9c3f80ed59a68c95 P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:31.824519 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 117) took 1031 ms. Trace:
I20260926 07:35:31.824782 11752 rpcz_store.cc:276] 0926 07:35:30.793187 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:30.793318 (+ 131us) service_pool.cc:227] Handling call
0926 07:35:31.646279 (+852961us) ts_tablet_manager.cc:1145] Deleting tablet 509f607b089c44bc9c3f80ed59a68c95
0926 07:35:31.646320 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:31.815629 (+169309us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:31.821576 (+ 5947us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:31.824470 (+ 2894us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":852445}
I20260926 07:35:31.825937 11752 tablet_replica.cc:330] T 1e1cea4523a64428aac03ca0bdf708c7 P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:31.826615 11752 raft_consensus.cc:2229] T 1e1cea4523a64428aac03ca0bdf708c7 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:31.827375 11752 raft_consensus.cc:2258] T 1e1cea4523a64428aac03ca0bdf708c7 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:31.827591 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet 509f607b089c44bc9c3f80ed59a68c95 (table default.Table_65217d931d594a70b45c7d5c486930ba [id=b0949f4718f8482881854a65f89d7421]) successfully deleted
I20260926 07:35:31.828896 11752 ts_tablet_manager.cc:1905] T 1e1cea4523a64428aac03ca0bdf708c7 P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:31.804483 12141 tablet.cc:1722] T 4826318c83524f41840e770f33bd3696 P e9bfebb34fa14cd2ad4843fffbe5c54e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260926 07:35:31.845791 12168 log.cc:1196] T 034d508f0a164f3ea8308ea2d28818c4 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/034d508f0a164f3ea8308ea2d28818c4
I20260926 07:35:31.846933 12168 ts_tablet_manager.cc:1939] T 034d508f0a164f3ea8308ea2d28818c4 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:31.848426 11113 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_cd53ff22f9444f688e341de983e3e145" } modify_external_catalogs: true
I20260926 07:35:31.849489 11113 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_cd53ff22f9444f688e341de983e3e145" } modify_external_catalogs: true
I20260926 07:35:31.855459 12154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249: Bootstrap starting.
I20260926 07:35:31.858777 11582 tablet_service.cc:1467] Processing CreateTablet for tablet f7459bf97b7f4b648d8ca6218cd72914 (DEFAULT_TABLE table=default.Table_170efaa4a249400e8c1d772a5ac0ec54 [id=e3d379a69ef94c6a990d3dfbea076591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:31.859854 11582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7459bf97b7f4b648d8ca6218cd72914. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:31.860313 11445 tablet_service.cc:1467] Processing CreateTablet for tablet f7459bf97b7f4b648d8ca6218cd72914 (DEFAULT_TABLE table=default.Table_170efaa4a249400e8c1d772a5ac0ec54 [id=e3d379a69ef94c6a990d3dfbea076591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:31.861706 11117 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 05f071b4db7847a5adbd25a71cac3ea7
I20260926 07:35:31.861759 11445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7459bf97b7f4b648d8ca6218cd72914. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:31.863037 11117 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:44836 (request call id 204) took 1977 ms. Trace:
I20260926 07:35:31.868369 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet 05f071b4db7847a5adbd25a71cac3ea7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:30 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:31.869539 11292 tablet_service.cc:1514] Processing DeleteTablet for tablet 05f071b4db7847a5adbd25a71cac3ea7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:30 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:31.863300 11117 rpcz_store.cc:276] 0926 07:35:29.885450 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.960320 (+ 74870us) service_pool.cc:227] Handling call
0926 07:35:30.095829 (+135509us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812376466 00007fb80feb7b34 00007fb80feb7ac2 00007fb80feb79f1 00007fb80feb796c 00007fb80feb6c22 00007fb80e15594c 00007fb80e154d86
0926 07:35:30.097489 (+ 1660us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0926 07:35:30.784672 (+687183us) catalog_manager.cc:2752] Modifying in-memory table state
0926 07:35:30.784803 (+ 131us) catalog_manager.cc:2761] Locking tablets
0926 07:35:31.074947 (+290144us) catalog_manager.cc:2775] Removing table and tablets from system table
0926 07:35:31.417830 (+342883us) catalog_manager.cc:2793] Removing table from by-name map
0926 07:35:31.418054 (+ 224us) catalog_manager.cc:2809] Committing in-memory state
0926 07:35:31.861624 (+443570us) catalog_manager.cc:2813] Aborting table tasks
0926 07:35:31.862989 (+ 1365us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:31.392143 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:31.392491 (+ 348us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:31.392513 (+ 22us) write_op.cc:435] Acquired schema lock
0926 07:35:31.392524 (+ 11us) tablet.cc:660] Decoding operations
0926 07:35:31.392834 (+ 310us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:31.392886 (+ 52us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:31.392918 (+ 32us) tablet.cc:683] Acquiring locks for 2 operations
0926 07:35:31.393236 (+ 318us) tablet.cc:699] Row locks acquired
0926 07:35:31.393256 (+ 20us) write_op.cc:260] PREPARE: finished
0926 07:35:31.393367 (+ 111us) write_op.cc:270] Start()
0926 07:35:31.393461 (+ 94us) write_op.cc:276] Timestamp: P: 1790408131393296 usec, L: 0
0926 07:35:31.393481 (+ 20us) op_driver.cc:331] REPLICATION: starting
0926 07:35:31.394057 (+ 576us) log.cc:842] Serialized 768 byte log entry
0926 07:35:31.414994 (+ 20937us) op_driver.cc:444] REPLICATION: finished
0926 07:35:31.415325 (+ 331us) write_op.cc:301] APPLY: starting
0926 07:35:31.415373 (+ 48us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:31.415468 (+ 95us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:31.415558 (+ 90us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:31.415788 (+ 230us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:31.415911 (+ 123us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0926 07:35:31.415950 (+ 39us) write_op.cc:312] APPLY: finished
0926 07:35:31.416737 (+ 787us) log.cc:842] Serialized 29 byte log entry
0926 07:35:31.416870 (+ 133us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:31.416934 (+ 64us) write_op.cc:454] Released schema lock
0926 07:35:31.416966 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:31.417132 (+ 166us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:31.417156 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":306130816,"child_traces":[["op",{"apply.queue_time_us":179,"apply.run_cpu_time_us":1404,"apply.run_wall_time_us":1987,"num_ops":2,"prepare.queue_time_us":315439,"prepare.run_cpu_time_us":2249,"prepare.run_wall_time_us":43075,"raft.queue_time_us":77,"raft.run_cpu_time_us":532,"raft.run_wall_time_us":531,"replication_time_us":21440}]]}
I20260926 07:35:31.869364 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet 05f071b4db7847a5adbd25a71cac3ea7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:30 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:31.868821 12175 tablet.cc:1722] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260926 07:35:31.877345 11746 tablet_bootstrap.cc:492] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:31.877724 11746 ts_tablet_manager.cc:1397] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.187s user 0.011s sys 0.001s
I20260926 07:35:31.882302 11292 tablet_service.cc:1467] Processing CreateTablet for tablet 8392ebd0a4b84c2c94c3fe8745ea1ad7 (DEFAULT_TABLE table=default.Table_26c01a15748a462e97fc9d5e934e51bf [id=3c2c877c4ddd4a2db31407350038576d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:31.884845 11583 tablet_service.cc:1467] Processing CreateTablet for tablet 8392ebd0a4b84c2c94c3fe8745ea1ad7 (DEFAULT_TABLE table=default.Table_26c01a15748a462e97fc9d5e934e51bf [id=3c2c877c4ddd4a2db31407350038576d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:31.885600 11290 tablet_service.cc:1467] Processing CreateTablet for tablet 8983eb4db03c4073b94e205ac83ff098 (DEFAULT_TABLE table=default.Table_db30b8ff3c7445d88ef57bc878b2051a [id=3c3ba3b39f2443cdb0bff709ee447727]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:31.885885 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8392ebd0a4b84c2c94c3fe8745ea1ad7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:31.886606 11754 tablet_bootstrap.cc:492] T 3f581510fe624127a4d27d302f5be614 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:31.886966 11754 ts_tablet_manager.cc:1397] T 3f581510fe624127a4d27d302f5be614 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.093s user 0.005s sys 0.006s
I20260926 07:35:31.890770 11754 raft_consensus.cc:348] T 3f581510fe624127a4d27d302f5be614 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:31.891469 11754 raft_consensus.cc:374] T 3f581510fe624127a4d27d302f5be614 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:31.891706 11754 raft_consensus.cc:729] T 3f581510fe624127a4d27d302f5be614 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:31.892045 11116 master_service.cc:946] ReplaceTablet: received request to replace tablet c81a59361a6e42b7962b3e027c4a087f from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:31.892302 11754 consensus_queue.cc:260] T 3f581510fe624127a4d27d302f5be614 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:31.894095 11754 ts_tablet_manager.cc:1428] T 3f581510fe624127a4d27d302f5be614 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260926 07:35:31.900080 11754 tablet_bootstrap.cc:492] T dbd4f14abfea4799b75c48531412fc20 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:31.896874 11756 tablet_bootstrap.cc:492] T 0eae53ce278749208492815b918582f3 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:31.916000 11754 tablet_bootstrap.cc:654] T dbd4f14abfea4799b75c48531412fc20 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:31.917111 11440 tablet_service.cc:1467] Processing CreateTablet for tablet 8392ebd0a4b84c2c94c3fe8745ea1ad7 (DEFAULT_TABLE table=default.Table_26c01a15748a462e97fc9d5e934e51bf [id=3c2c877c4ddd4a2db31407350038576d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:31.917809 11286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7459bf97b7f4b648d8ca6218cd72914. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:31.918502 11440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8392ebd0a4b84c2c94c3fe8745ea1ad7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:31.919515 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet 034d508f0a164f3ea8308ea2d28818c4 (table default.Table_53678f9c4c01403eb2d846d3242be0aa [id=db19d49c9c29488e89f7280e91a11713]) successfully deleted
I20260926 07:35:31.919871 12168 tablet_replica.cc:330] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:31.920620 11292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8392ebd0a4b84c2c94c3fe8745ea1ad7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:31.924564 11290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8983eb4db03c4073b94e205ac83ff098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:31.925827 11746 raft_consensus.cc:348] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.926491 11746 raft_consensus.cc:374] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:31.926753 11442 tablet_service.cc:1467] Processing CreateTablet for tablet 8983eb4db03c4073b94e205ac83ff098 (DEFAULT_TABLE table=default.Table_db30b8ff3c7445d88ef57bc878b2051a [id=3c3ba3b39f2443cdb0bff709ee447727]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:31.928145 11442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8983eb4db03c4073b94e205ac83ff098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:31.932660 12168 raft_consensus.cc:2229] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:31.933737 12168 raft_consensus.cc:2258] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:31.934849 11590 tablet_service.cc:1467] Processing CreateTablet for tablet 8983eb4db03c4073b94e205ac83ff098 (DEFAULT_TABLE table=default.Table_db30b8ff3c7445d88ef57bc878b2051a [id=3c3ba3b39f2443cdb0bff709ee447727]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:31.935264 11288 tablet_service.cc:1467] Processing CreateTablet for tablet f4aa8bef4bc64953ae2c573238838469 (DEFAULT_TABLE table=default.Table_c19ab6c510e64f6bb846aa3dc6f1c2fe [id=df787105db4c48d9bd29faca9a0c7a15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:31.935768 11590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8983eb4db03c4073b94e205ac83ff098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:31.936223 11288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4aa8bef4bc64953ae2c573238838469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:31.936679 12168 ts_tablet_manager.cc:1905] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:31.940707 11871 ts_tablet_manager.cc:1918] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:31.941044 11871 log.cc:1196] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/9a255ef54f7e410ca8a779e5d7a118ca
I20260926 07:35:31.941596 11871 ts_tablet_manager.cc:1939] T 9a255ef54f7e410ca8a779e5d7a118ca P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:31.941787 12125 raft_consensus.cc:482] T 3f581510fe624127a4d27d302f5be614 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:31.942173 12125 raft_consensus.cc:504] T 3f581510fe624127a4d27d302f5be614 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:31.943836 12125 leader_election.cc:288] T 3f581510fe624127a4d27d302f5be614 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:31.945843 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f581510fe624127a4d27d302f5be614" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:31.926726 11746 raft_consensus.cc:729] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:31.948392 11116 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_676572a0e9ac48f5a9ead303605b7ffa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:31.949651 11462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f581510fe624127a4d27d302f5be614" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
W20260926 07:35:31.949834 11116 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_676572a0e9ac48f5a9ead303605b7ffa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:31.952587 11746 consensus_queue.cc:260] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:31.958796 11435 tablet_service.cc:1467] Processing CreateTablet for tablet f4aa8bef4bc64953ae2c573238838469 (DEFAULT_TABLE table=default.Table_c19ab6c510e64f6bb846aa3dc6f1c2fe [id=df787105db4c48d9bd29faca9a0c7a15]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 07:35:31.950964 11546 leader_election.cc:341] T 3f581510fe624127a4d27d302f5be614 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:31.959812 11435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4aa8bef4bc64953ae2c573238838469. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:31.960062 11545 leader_election.cc:341] T 3f581510fe624127a4d27d302f5be614 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:31.960799 11545 leader_election.cc:302] T 3f581510fe624127a4d27d302f5be614 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:31.961957 12134 raft_consensus.cc:2726] T 3f581510fe624127a4d27d302f5be614 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:35:31.963099 11112 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.3:53335 (request call id 28) took 2214 ms (client timeout 1000 ms). Trace:
I20260926 07:35:31.965147 11752 ts_tablet_manager.cc:1918] T 1e1cea4523a64428aac03ca0bdf708c7 P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:31.965382 11589 tablet_service.cc:1467] Processing CreateTablet for tablet f4aa8bef4bc64953ae2c573238838469 (DEFAULT_TABLE table=default.Table_c19ab6c510e64f6bb846aa3dc6f1c2fe [id=df787105db4c48d9bd29faca9a0c7a15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:31.965582 11752 log.cc:1196] T 1e1cea4523a64428aac03ca0bdf708c7 P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/1e1cea4523a64428aac03ca0bdf708c7
I20260926 07:35:31.966305 11589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4aa8bef4bc64953ae2c573238838469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:31.967872 11289 tablet_service.cc:1467] Processing CreateTablet for tablet 854ac84f21304d57818dd052464127b8 (DEFAULT_TABLE table=default.Table_c24b830c62a24ff989bceed92112ed52 [id=83482df6f10341778ca79919c84c5fc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:31.954521 11746 ts_tablet_manager.cc:1428] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.077s user 0.005s sys 0.000s
I20260926 07:35:31.971990 11746 tablet_bootstrap.cc:492] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:31.974864 11439 tablet_service.cc:1467] Processing CreateTablet for tablet 854ac84f21304d57818dd052464127b8 (DEFAULT_TABLE table=default.Table_c24b830c62a24ff989bceed92112ed52 [id=83482df6f10341778ca79919c84c5fc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:31.975847 11439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 854ac84f21304d57818dd052464127b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:31.979858 12035 raft_consensus.cc:482] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:31.980301 12035 raft_consensus.cc:504] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:31.978489 11112 rpcz_store.cc:269] 0926 07:35:29.748596 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.776625 (+ 28029us) service_pool.cc:227] Handling call
0926 07:35:29.847794 (+ 71169us) spinlock_profiling.cc:243] Waited 67.6 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812374c3e 00007fb80feabfb4 00007fb80feabf42 00007fb80feabe71 00007fb80feabdec 00007fb80feab0a2 00007fb80e15594c 00007fb80e154d86
0926 07:35:31.963023 (+2115229us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:30.676987 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:30.677313 (+ 326us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:30.677333 (+ 20us) write_op.cc:435] Acquired schema lock
0926 07:35:30.677345 (+ 12us) tablet.cc:660] Decoding operations
0926 07:35:30.677587 (+ 242us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:30.677619 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:30.677640 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0926 07:35:30.677758 (+ 118us) tablet.cc:699] Row locks acquired
0926 07:35:30.677770 (+ 12us) write_op.cc:260] PREPARE: finished
0926 07:35:30.677866 (+ 96us) write_op.cc:270] Start()
0926 07:35:30.677937 (+ 71us) write_op.cc:276] Timestamp: P: 1790408130677819 usec, L: 0
0926 07:35:30.677952 (+ 15us) op_driver.cc:331] REPLICATION: starting
0926 07:35:30.678367 (+ 415us) log.cc:842] Serialized 530 byte log entry
0926 07:35:30.762057 (+ 83690us) op_driver.cc:444] REPLICATION: finished
0926 07:35:30.765585 (+ 3528us) write_op.cc:301] APPLY: starting
0926 07:35:30.765628 (+ 43us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:30.765711 (+ 83us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:30.765785 (+ 74us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:30.765913 (+ 128us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:30.766020 (+ 107us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:30.766058 (+ 38us) write_op.cc:312] APPLY: finished
0926 07:35:30.766183 (+ 125us) log.cc:842] Serialized 23 byte log entry
0926 07:35:30.766284 (+ 101us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:30.766331 (+ 47us) write_op.cc:454] Released schema lock
0926 07:35:30.766364 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:30.766506 (+ 142us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:30.766528 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":12,"spinlock_wait_cycles":155421440,"child_traces":[["op",{"apply.queue_time_us":3425,"apply.run_cpu_time_us":1066,"apply.run_wall_time_us":1066,"num_ops":1,"prepare.queue_time_us":123808,"prepare.run_cpu_time_us":1629,"prepare.run_wall_time_us":19348,"replication_time_us":84027,"thread_start_us":96611,"threads_started":1}]]}
I20260926 07:35:31.981832 11112 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_c24b830c62a24ff989bceed92112ed52" } new_table_name: "default.Table_52e8769a2e55412eb074de5c4f541ba6" modify_external_catalogs: true
I20260926 07:35:31.982133 12035 leader_election.cc:288] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:31.984346 11444 tablet_service.cc:1514] Processing DeleteTablet for tablet 509f607b089c44bc9c3f80ed59a68c95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:31.985090 11437 tablet_service.cc:1514] Processing DeleteTablet for tablet 640d21586bad4070957cff61bd174fd2 with delete_type TABLET_DATA_DELETED (replaced by tablet 6d54a5c32da5444ca0a2939461f3a0f6) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:31.985672 11436 tablet_service.cc:1514] Processing DeleteTablet for tablet d6fc6178f1e54c3984a20873864233cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:26 UTC) from {username='slave'} at 127.0.0.1:40244
W20260926 07:35:31.986397 11119 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.1:34359 (request call id 43) took 1505 ms (client timeout 1000 ms). Trace:
I20260926 07:35:31.988672 11756 tablet_bootstrap.cc:654] T 0eae53ce278749208492815b918582f3 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:31.992389 11295 tablet_service.cc:1514] Processing DeleteTablet for tablet 640d21586bad4070957cff61bd174fd2 with delete_type TABLET_DATA_DELETED (replaced by tablet 6d54a5c32da5444ca0a2939461f3a0f6) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:31.992626 11285 tablet_service.cc:1514] Processing DeleteTablet for tablet d6fc6178f1e54c3984a20873864233cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:26 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:31.985091 11444 tablet_service.cc:1514] Processing DeleteTablet for tablet 55ad1d96e2314d1eb0a717a3e9ff31bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:31.993620 11296 tablet_service.cc:1514] Processing DeleteTablet for tablet 04be706330924282961054c5835cec73 with delete_type TABLET_DATA_DELETED (replaced by tablet 21919d22b1d24546911d36384b01d9cc) from {username='slave'} at 127.0.0.1:52070
W20260926 07:35:31.993767 11111 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.2:33317 (request call id 46) took 1180 ms (client timeout 1000 ms). Trace:
I20260926 07:35:31.993835 12141 tablet.cc:1722] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260926 07:35:31.994081 11111 rpcz_store.cc:269] 0926 07:35:30.813730 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:30.823955 (+ 10225us) service_pool.cc:227] Handling call
0926 07:35:30.898910 (+ 74955us) spinlock_profiling.cc:243] Waited 71.2 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812374c3e 00007fb80feabfb4 00007fb80feabf42 00007fb80feabe71 00007fb80feabdec 00007fb80feab0a2 00007fb80e15594c 00007fb80e154d86
0926 07:35:31.993708 (+1094798us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":11,"spinlock_wait_cycles":163822848}
I20260926 07:35:31.996745 12108 tablet.cc:1722] T a55e54d492214039b0294f954b5ef390 P 22f3902d01584936b20437606cb86dd1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 07:35:31.998153 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0218d2a727f942e48d770878e6bf4a54" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:31.998406 11581 tablet_service.cc:1514] Processing DeleteTablet for tablet a8b629a4aba045b697e33bbebecb05b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:31.998672 11591 tablet_service.cc:1514] Processing DeleteTablet for tablet e06ed286144941269b9a0c0c60bff0ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:24 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:31.998780 11608 raft_consensus.cc:2454] T 0218d2a727f942e48d770878e6bf4a54 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 0.
I20260926 07:35:31.999562 11591 tablet_service.cc:1514] Processing DeleteTablet for tablet 509f607b089c44bc9c3f80ed59a68c95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:32.000437 11591 tablet_service.cc:1514] Processing DeleteTablet for tablet 640d21586bad4070957cff61bd174fd2 with delete_type TABLET_DATA_DELETED (replaced by tablet 6d54a5c32da5444ca0a2939461f3a0f6) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:32.000440 11249 leader_election.cc:302] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:32.001245 12069 raft_consensus.cc:2781] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:32.001303 11591 tablet_service.cc:1514] Processing DeleteTablet for tablet 1620b507f07c446c8c1811895a097409 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:26 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:32.001566 12069 raft_consensus.cc:482] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:32.001861 12069 raft_consensus.cc:3037] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:32.007094 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0218d2a727f942e48d770878e6bf4a54" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:32.007808 11463 raft_consensus.cc:2454] T 0218d2a727f942e48d770878e6bf4a54 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 0.
W20260926 07:35:31.986698 11119 rpcz_store.cc:269] 0926 07:35:30.481282 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:30.517284 (+ 36002us) service_pool.cc:227] Handling call
0926 07:35:31.986338 (+1469054us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:30.937245 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:30.937574 (+ 329us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:30.937594 (+ 20us) write_op.cc:435] Acquired schema lock
0926 07:35:30.937605 (+ 11us) tablet.cc:660] Decoding operations
0926 07:35:30.937832 (+ 227us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:30.937869 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:30.937888 (+ 19us) tablet.cc:683] Acquiring locks for 1 operations
0926 07:35:30.938013 (+ 125us) tablet.cc:699] Row locks acquired
0926 07:35:30.938026 (+ 13us) write_op.cc:260] PREPARE: finished
0926 07:35:30.938120 (+ 94us) write_op.cc:270] Start()
0926 07:35:30.938188 (+ 68us) write_op.cc:276] Timestamp: P: 1790408130938050 usec, L: 0
0926 07:35:30.938203 (+ 15us) op_driver.cc:331] REPLICATION: starting
0926 07:35:30.938634 (+ 431us) log.cc:842] Serialized 530 byte log entry
0926 07:35:31.070482 (+131848us) op_driver.cc:444] REPLICATION: finished
0926 07:35:31.071588 (+ 1106us) write_op.cc:301] APPLY: starting
0926 07:35:31.071637 (+ 49us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:31.071722 (+ 85us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:31.071842 (+ 120us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:31.072014 (+ 172us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:31.072168 (+ 154us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:31.072217 (+ 49us) write_op.cc:312] APPLY: finished
0926 07:35:31.072388 (+ 171us) log.cc:842] Serialized 23 byte log entry
0926 07:35:31.073003 (+ 615us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:31.073055 (+ 52us) write_op.cc:454] Released schema lock
0926 07:35:31.073084 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:31.073255 (+ 171us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:31.073278 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":955,"apply.run_cpu_time_us":1382,"apply.run_wall_time_us":1857,"num_ops":1,"prepare.queue_time_us":159921,"prepare.run_cpu_time_us":1621,"prepare.run_wall_time_us":2692,"raft.queue_time_us":73,"raft.run_cpu_time_us":10,"raft.run_wall_time_us":10,"replication_time_us":132208}]]}
I20260926 07:35:32.015246 11113 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 6d54a5c32da5444ca0a2939461f3a0f6
I20260926 07:35:32.016834 11296 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d54a5c32da5444ca0a2939461f3a0f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:31 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:32.017441 11444 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d54a5c32da5444ca0a2939461f3a0f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:31 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:32.018473 11591 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d54a5c32da5444ca0a2939461f3a0f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:31 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:32.020829 11660 heartbeater.cc:503] Master 127.10.171.62:37629 was elected leader, sending a full tablet report...
I20260926 07:35:32.024565 11370 heartbeater.cc:347] Connected to a master server at 127.10.171.62:37629
I20260926 07:35:32.038982 11754 tablet_bootstrap.cc:492] T dbd4f14abfea4799b75c48531412fc20 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:32.039340 11754 ts_tablet_manager.cc:1397] T dbd4f14abfea4799b75c48531412fc20 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.139s user 0.004s sys 0.004s
I20260926 07:35:32.041384 11754 raft_consensus.cc:348] T dbd4f14abfea4799b75c48531412fc20 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:32.041993 11754 raft_consensus.cc:374] T dbd4f14abfea4799b75c48531412fc20 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:32.042224 11754 raft_consensus.cc:729] T dbd4f14abfea4799b75c48531412fc20 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:32.043681 11746 tablet_bootstrap.cc:654] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:32.047024 11591 tablet_service.cc:1467] Processing CreateTablet for tablet 854ac84f21304d57818dd052464127b8 (DEFAULT_TABLE table=default.Table_c24b830c62a24ff989bceed92112ed52 [id=83482df6f10341778ca79919c84c5fc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:31.966287 11752 ts_tablet_manager.cc:1939] T 1e1cea4523a64428aac03ca0bdf708c7 P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:32.050498 11754 consensus_queue.cc:260] T dbd4f14abfea4799b75c48531412fc20 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:32.052956 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 118) took 1259 ms. Trace:
I20260926 07:35:32.055971 11119 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_8b58d5bf0cf74a2f812cf5f7602515b8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:32.057479 11119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8b58d5bf0cf74a2f812cf5f7602515b8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:32.063618 11111 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_e0dcc535d6e54d679a9444cf6cec1445" } new_table_name: "default.Table_4d993e44ddc44452a2be1c319f81f9e8" modify_external_catalogs: true
I20260926 07:35:32.073482 11754 ts_tablet_manager.cc:1428] T dbd4f14abfea4799b75c48531412fc20 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.034s user 0.006s sys 0.001s
I20260926 07:35:32.074213 11754 tablet_bootstrap.cc:492] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:32.074280 11591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 854ac84f21304d57818dd052464127b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:32.081338 12183 tablet.cc:1722] T a55e54d492214039b0294f954b5ef390 P 6070e9e3d1234a9d83e49431416afade: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 07:35:32.093205 11120 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_8d6e8e19741e40d0a01d30a0f21fef8a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:32.096792 11120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8d6e8e19741e40d0a01d30a0f21fef8a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:32.099052 11113 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_2f42e2e9196745bb942de35a1373e040" } modify_external_catalogs: true
I20260926 07:35:32.099648 11113 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_2f42e2e9196745bb942de35a1373e040" } modify_external_catalogs: true
I20260926 07:35:32.029531 11289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 854ac84f21304d57818dd052464127b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:32.032986 11288 tablet_service.cc:1514] Processing DeleteTablet for tablet a8b629a4aba045b697e33bbebecb05b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:32.036844 12069 raft_consensus.cc:504] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.113582 12069 leader_election.cc:288] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Requested vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:32.038997 11756 tablet_bootstrap.cc:492] T 0eae53ce278749208492815b918582f3 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:32.114233 11756 ts_tablet_manager.cc:1397] T 0eae53ce278749208492815b918582f3 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.218s user 0.009s sys 0.000s
I20260926 07:35:32.099512 11286 tablet_service.cc:1467] Processing CreateTablet for tablet 6d54a5c32da5444ca0a2939461f3a0f6 (DEFAULT_TABLE table=default.Table_cd53ff22f9444f688e341de983e3e145 [id=fa5001d12cb14306b0a5ab0f0c79bc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:32.118700 11286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d54a5c32da5444ca0a2939461f3a0f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:32.054702 11292 tablet_service.cc:1467] Processing CreateTablet for tablet 393da59b3671473ca9714d3406dd1388 (DEFAULT_TABLE table=default.Table_54b98a70017749499ab6c312bd7e6255 [id=3895daba9d194983a7018f02aa7b02a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:32.124756 11292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 393da59b3671473ca9714d3406dd1388. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:32.130836 11756 raft_consensus.cc:348] T 0eae53ce278749208492815b918582f3 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.131479 11756 raft_consensus.cc:374] T 0eae53ce278749208492815b918582f3 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:32.131698 11756 raft_consensus.cc:729] T 0eae53ce278749208492815b918582f3 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:32.135049 11756 consensus_queue.cc:260] T 0eae53ce278749208492815b918582f3 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:32.144955 11111 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5f2ef7ebe6e144529cffd699180d5be8 has no consensus state
I20260926 07:35:32.145167 11756 ts_tablet_manager.cc:1428] T 0eae53ce278749208492815b918582f3 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.031s user 0.003s sys 0.002s
I20260926 07:35:32.147059 11756 tablet_bootstrap.cc:492] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:32.055801 11445 tablet_service.cc:1467] Processing CreateTablet for tablet 393da59b3671473ca9714d3406dd1388 (DEFAULT_TABLE table=default.Table_54b98a70017749499ab6c312bd7e6255 [id=3895daba9d194983a7018f02aa7b02a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:32.148753 11445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 393da59b3671473ca9714d3406dd1388. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:32.100621 11442 tablet_service.cc:1467] Processing CreateTablet for tablet 6d54a5c32da5444ca0a2939461f3a0f6 (DEFAULT_TABLE table=default.Table_cd53ff22f9444f688e341de983e3e145 [id=fa5001d12cb14306b0a5ab0f0c79bc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:32.152709 11442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d54a5c32da5444ca0a2939461f3a0f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:32.146251 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0218d2a727f942e48d770878e6bf4a54" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:32.156567 11463 raft_consensus.cc:3037] T 0218d2a727f942e48d770878e6bf4a54 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:32.171831 11463 raft_consensus.cc:2454] T 0218d2a727f942e48d770878e6bf4a54 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 1.
I20260926 07:35:32.172930 11247 leader_election.cc:302] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
W20260926 07:35:32.173188 11118 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.3:53335 (request call id 29) took 1368 ms (client timeout 1000 ms). Trace:
I20260926 07:35:32.173661 12069 raft_consensus.cc:2781] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election won for term 1
W20260926 07:35:32.173470 11118 rpcz_store.cc:269] 0926 07:35:30.804779 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:30.811846 (+ 7067us) service_pool.cc:227] Handling call
0926 07:35:30.930985 (+119139us) spinlock_profiling.cc:243] Waited 93.9 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812374c3e 00007fb80feabfb4 00007fb80feabf42 00007fb80feabe71 00007fb80feabdec 00007fb80feab0a2 00007fb80e15594c 00007fb80e154d86
0926 07:35:32.173126 (+1242141us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":20,"spinlock_wait_cycles":215924096}
I20260926 07:35:32.174116 12069 raft_consensus.cc:686] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Becoming Leader. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Running, Role: LEADER
I20260926 07:35:32.174976 12069 consensus_queue.cc:237] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.099040 11582 tablet_service.cc:1467] Processing CreateTablet for tablet 393da59b3671473ca9714d3406dd1388 (DEFAULT_TABLE table=default.Table_54b98a70017749499ab6c312bd7e6255 [id=3895daba9d194983a7018f02aa7b02a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:32.184864 11582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 393da59b3671473ca9714d3406dd1388. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:32.102648 12133 raft_consensus.cc:482] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:32.187855 12133 raft_consensus.cc:504] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.189723 12133 leader_election.cc:288] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:32.106004 11754 tablet_bootstrap.cc:654] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:32.156472 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0218d2a727f942e48d770878e6bf4a54" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:32.166988 12134 raft_consensus.cc:482] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:32.169320 11581 tablet_service.cc:1514] Processing DeleteTablet for tablet c81a59361a6e42b7962b3e027c4a087f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:28 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:32.190605 11608 raft_consensus.cc:3037] T 0218d2a727f942e48d770878e6bf4a54 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:32.169322 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet 55ad1d96e2314d1eb0a717a3e9ff31bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:32.188584 11756 tablet_bootstrap.cc:654] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:32.183823 11752 rpcz_store.cc:276] 0926 07:35:30.793530 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:30.793923 (+ 393us) service_pool.cc:227] Handling call
0926 07:35:31.825815 (+1031892us) ts_tablet_manager.cc:1145] Deleting tablet 1e1cea4523a64428aac03ca0bdf708c7
0926 07:35:31.825848 (+ 33us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:31.838868 (+ 13020us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:31.964956 (+126088us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:32.052905 (+ 87949us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1031445}
I20260926 07:35:32.193230 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet 1e1cea4523a64428aac03ca0bdf708c7 (table default.Table_7846bbd3f8cd4fbbb54ce45d2e3ce6b9 [id=41849849f4ae4a9396ccda32d995e21a]) successfully deleted
I20260926 07:35:32.194398 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 119) took 1375 ms. Trace:
I20260926 07:35:32.194756 11752 rpcz_store.cc:276] 0926 07:35:30.818732 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:30.819503 (+ 771us) service_pool.cc:227] Handling call
0926 07:35:32.194053 (+1374550us) ts_tablet_manager.cc:1145] Deleting tablet 21919d22b1d24546911d36384b01d9cc
0926 07:35:32.194095 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:32.194327 (+ 232us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1374013}
W20260926 07:35:32.195920 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 21919d22b1d24546911d36384b01d9cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 21919d22b1d24546911d36384b01d9cc already in progress: creating tablet
I20260926 07:35:32.196446 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e28ca159d8349748812825ac1947c51" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:32.196938 11752 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 120) took 1286 ms. Trace:
I20260926 07:35:32.197002 12164 raft_consensus.cc:482] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260926 07:35:32.197656 11545 leader_election.cc:341] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:32.197346 11752 rpcz_store.cc:276] 0926 07:35:30.910310 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:30.910458 (+ 148us) service_pool.cc:227] Handling call
0926 07:35:32.196638 (+1286180us) ts_tablet_manager.cc:1145] Deleting tablet 04180ace5d4548dfbcd226bf229d5c26
0926 07:35:32.196677 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:32.196890 (+ 213us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1285654}
I20260926 07:35:32.197463 12164 raft_consensus.cc:504] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:32.198330 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 04180ace5d4548dfbcd226bf229d5c26 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 04180ace5d4548dfbcd226bf229d5c26 already in progress: creating tablet
I20260926 07:35:32.199263 12164 leader_election.cc:288] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:32.200546 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eae53ce278749208492815b918582f3" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:32.201076 11313 raft_consensus.cc:2454] T 0eae53ce278749208492815b918582f3 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:32.202261 11401 leader_election.cc:302] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:32.203082 12164 raft_consensus.cc:2781] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:32.203369 12164 raft_consensus.cc:482] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:32.203626 12164 raft_consensus.cc:3037] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:32.210350 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 120) took 2548 ms. Trace:
I20260926 07:35:32.215850 11288 tablet_service.cc:1467] Processing CreateTablet for tablet 89ddc16f2635473da43fe21676a947fa (DEFAULT_TABLE table=default.Table_d0380e9d3f26439bbc91cf476aeae69f [id=19875953f2b6484f934503404af57593]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:32.215889 11435 tablet_service.cc:1467] Processing CreateTablet for tablet 89ddc16f2635473da43fe21676a947fa (DEFAULT_TABLE table=default.Table_d0380e9d3f26439bbc91cf476aeae69f [id=19875953f2b6484f934503404af57593]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:32.216951 11288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89ddc16f2635473da43fe21676a947fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:32.217001 11435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89ddc16f2635473da43fe21676a947fa. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:32.223717 11112 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 854ac84f21304d57818dd052464127b8 consensus state has no leader
I20260926 07:35:32.227460 11116 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_f6f34cb43e5c462aa22636e6daaf274e" } modify_external_catalogs: true
I20260926 07:35:32.227427 12164 raft_consensus.cc:504] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.228029 11116 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_f6f34cb43e5c462aa22636e6daaf274e" } modify_external_catalogs: true
I20260926 07:35:32.231386 11444 tablet_service.cc:1467] Processing CreateTablet for tablet 31e135bc19994295a1fc1dcf39cca5cb (DEFAULT_TABLE table=default.Table_19dc9207bc6249c98efc072fe24ba570 [id=278738ede0a442d58aa9051c90a718c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:32.231542 11111 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_95dfccbf3504491bb70c84a9ac9038d1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:32.232374 11444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31e135bc19994295a1fc1dcf39cca5cb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:32.233141 11111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_95dfccbf3504491bb70c84a9ac9038d1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:32.240244 12164 leader_election.cc:288] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:32.190416 12134 raft_consensus.cc:504] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:32.246493 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eae53ce278749208492815b918582f3" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:32.247004 11609 raft_consensus.cc:3037] T 0eae53ce278749208492815b918582f3 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:32.247442 12134 leader_election.cc:288] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:32.248600 11871 rpcz_store.cc:276] 0926 07:35:29.661636 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.661744 (+ 108us) service_pool.cc:227] Handling call
0926 07:35:31.633552 (+1971808us) ts_tablet_manager.cc:1145] Deleting tablet 9a255ef54f7e410ca8a779e5d7a118ca
0926 07:35:31.633592 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:31.789706 (+156114us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b20000827b0. stack: 00007f9f9373c033 00007f9f9373bf30 00007f9f932bd534 00000000004db2b4 00000000004db249 00007f9f9b6057f1 00007f9f9b636663 00007f9f9b63dde2 00007f9f9b63dd61 00007f9f9b63dcf1 00007f9f9b63dcb1 00007f9f9b63ca1d 0000000000580784 00000000005805b9 00007f9f936c0b3f 00007f9f9aa341d6
0926 07:35:31.826388 (+ 36682us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:31.940515 (+114127us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:32.210283 (+269768us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":303367040,"tablet-delete.queue_time_us":1971262}
W20260926 07:35:32.249709 11545 leader_election.cc:341] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Not found: Tablet not found: afa901cdc6a84220bf36ee669a955b72
I20260926 07:35:32.250929 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 9a255ef54f7e410ca8a779e5d7a118ca (table default.Table_836a28372ec4439d98b1583a9b50d2c1 [id=800a96cbd0b34639bbe7ad317a640053]) successfully deleted
I20260926 07:35:32.251184 11871 tablet_replica.cc:330] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:32.253150 11583 tablet_service.cc:1467] Processing CreateTablet for tablet 6d54a5c32da5444ca0a2939461f3a0f6 (DEFAULT_TABLE table=default.Table_cd53ff22f9444f688e341de983e3e145 [id=fa5001d12cb14306b0a5ab0f0c79bc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:32.254205 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d54a5c32da5444ca0a2939461f3a0f6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:32.257432 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 30df9b4afb414c90af5f4fb0c4582f01 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 30df9b4afb414c90af5f4fb0c4582f01 already in progress: creating tablet
I20260926 07:35:32.257790 11581 tablet_service.cc:1467] Processing CreateTablet for tablet 89ddc16f2635473da43fe21676a947fa (DEFAULT_TABLE table=default.Table_d0380e9d3f26439bbc91cf476aeae69f [id=19875953f2b6484f934503404af57593]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:32.244513 11610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eae53ce278749208492815b918582f3" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:32.261497 11752 tablet_replica.cc:330] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:32.261837 11590 tablet_service.cc:1467] Processing CreateTablet for tablet 31e135bc19994295a1fc1dcf39cca5cb (DEFAULT_TABLE table=default.Table_19dc9207bc6249c98efc072fe24ba570 [id=278738ede0a442d58aa9051c90a718c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:32.262921 11752 raft_consensus.cc:2229] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:32.265914 11752 raft_consensus.cc:2258] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:32.272279 11609 raft_consensus.cc:2454] T 0eae53ce278749208492815b918582f3 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
W20260926 07:35:32.273151 11545 consensus_peers.cc:487] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade -> Peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Couldn't send request to peer 22f3902d01584936b20437606cb86dd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1e1cea4523a64428aac03ca0bdf708c7. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:32.278906 11590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31e135bc19994295a1fc1dcf39cca5cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:32.281822 11752 ts_tablet_manager.cc:1905] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:32.273303 11402 leader_election.cc:302] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:32.276754 11445 tablet_service.cc:1467] Processing CreateTablet for tablet 16ab928a487c4f499aba3b366d01d89b (DEFAULT_TABLE table=default.Table_0d113826d96240549bc980febd8776e0 [id=f9582c45829d4f10af98e245dd41a907]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:32.287187 12164 raft_consensus.cc:2781] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:32.287611 12164 raft_consensus.cc:686] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:32.288005 11445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16ab928a487c4f499aba3b366d01d89b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:32.288360 12164 consensus_queue.cc:237] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:32.289368 11106 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 5f2ef7ebe6e144529cffd699180d5be8 (table default.Table_4d993e44ddc44452a2be1c319f81f9e8 [id=b359877938204095b5ed3cfbe94997a3], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5f2ef7ebe6e144529cffd699180d5be8 has no consensus state
I20260926 07:35:32.263666 11746 tablet_bootstrap.cc:492] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:32.293321 11746 ts_tablet_manager.cc:1397] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.321s user 0.010s sys 0.000s
I20260926 07:35:32.296517 11581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89ddc16f2635473da43fe21676a947fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:32.295506 11746 raft_consensus.cc:348] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.299901 11746 raft_consensus.cc:374] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:32.303265 11588 tablet_service.cc:1467] Processing CreateTablet for tablet 16ab928a487c4f499aba3b366d01d89b (DEFAULT_TABLE table=default.Table_0d113826d96240549bc980febd8776e0 [id=f9582c45829d4f10af98e245dd41a907]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:32.304921 11754 tablet_bootstrap.cc:492] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:32.305867 11754 ts_tablet_manager.cc:1397] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.231s user 0.005s sys 0.013s
I20260926 07:35:32.300186 11746 raft_consensus.cc:729] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:32.310907 11754 raft_consensus.cc:348] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:32.311913 11754 raft_consensus.cc:374] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:32.312219 11754 raft_consensus.cc:729] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:32.313069 11588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16ab928a487c4f499aba3b366d01d89b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:32.313199 11754 consensus_queue.cc:260] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:32.330531 11754 ts_tablet_manager.cc:1428] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
I20260926 07:35:32.331879 11754 tablet_bootstrap.cc:492] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:32.335557 12134 raft_consensus.cc:482] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:32.336195 12134 raft_consensus.cc:504] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:32.338038 12134 leader_election.cc:288] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:32.339679 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c092aff8f2a041ccb09bc4b48e0f359c" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
W20260926 07:35:32.341528 11545 leader_election.cc:341] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:32.344230 11746 consensus_queue.cc:260] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.356626 11746 ts_tablet_manager.cc:1428] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.063s user 0.005s sys 0.002s
I20260926 07:35:32.364614 11746 tablet_bootstrap.cc:492] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:32.369798 11746 tablet_bootstrap.cc:654] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:32.377930 11117 catalog_manager.cc:5495] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade reported cstate change: term changed from 0 to 1, leader changed from <none> to 6070e9e3d1234a9d83e49431416afade (127.10.171.3). New cstate: current_term: 1 leader_uuid: "6070e9e3d1234a9d83e49431416afade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } health_report { overall_health: HEALTHY } } }
I20260926 07:35:32.384879 11112 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_02974e73af5f4521a34bcb9d2c75a84a" } new_table_name: "default.Table_86d0f0857c004fb8b506a9fde5aa3333" modify_external_catalogs: true
I20260926 07:35:32.393898 11746 tablet_bootstrap.cc:492] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:32.394312 11746 ts_tablet_manager.cc:1397] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.030s user 0.021s sys 0.006s
I20260926 07:35:32.397575 11754 tablet_bootstrap.cc:654] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:32.401729 11746 raft_consensus.cc:348] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.402364 11746 raft_consensus.cc:374] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:32.402601 11746 raft_consensus.cc:729] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:32.406750 11746 consensus_queue.cc:260] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.411151 11746 ts_tablet_manager.cc:1428] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.017s user 0.004s sys 0.001s
I20260926 07:35:32.428047 11746 tablet_bootstrap.cc:492] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:32.436523 11752 ts_tablet_manager.cc:1918] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:32.437340 11752 log.cc:1196] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/034d508f0a164f3ea8308ea2d28818c4
I20260926 07:35:32.438387 11752 ts_tablet_manager.cc:1939] T 034d508f0a164f3ea8308ea2d28818c4 P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:32.449587 11746 tablet_bootstrap.cc:654] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:32.451690 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet 034d508f0a164f3ea8308ea2d28818c4 (table default.Table_53678f9c4c01403eb2d846d3242be0aa [id=db19d49c9c29488e89f7280e91a11713]) successfully deleted
I20260926 07:35:32.453176 11752 tablet_replica.cc:330] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:32.455999 11752 raft_consensus.cc:2229] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:32.457173 11752 raft_consensus.cc:2258] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:32.461911 11660 heartbeater.cc:503] Master 127.10.171.62:37629 was elected leader, sending a full tablet report...
I20260926 07:35:32.462504 11752 ts_tablet_manager.cc:1905] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:32.391808 11119 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_47fd16cdaf2a4eedbfb8a02517543f80"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:32.485612 11119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_47fd16cdaf2a4eedbfb8a02517543f80 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:32.399544 11608 raft_consensus.cc:2454] T 0218d2a727f942e48d770878e6bf4a54 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 1.
I20260926 07:35:32.470104 12125 consensus_queue.cc:1035] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 07:35:32.479388 11871 raft_consensus.cc:2229] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [term 2 FOLLOWER]: Raft consensus shutting down.
W20260926 07:35:32.493301 11545 consensus_peers.cc:487] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade -> Peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Couldn't send request to peer 22f3902d01584936b20437606cb86dd1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:32.525854 11290 tablet_service.cc:1467] Processing CreateTablet for tablet 31e135bc19994295a1fc1dcf39cca5cb (DEFAULT_TABLE table=default.Table_19dc9207bc6249c98efc072fe24ba570 [id=278738ede0a442d58aa9051c90a718c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:32.546394 11871 raft_consensus.cc:2258] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade [term 2 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:32.510883 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eae53ce278749208492815b918582f3" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:32.560884 11316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e28ca159d8349748812825ac1947c51" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:32.561486 11316 raft_consensus.cc:2454] T 3e28ca159d8349748812825ac1947c51 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 0.
I20260926 07:35:32.562387 11296 tablet_service.cc:1467] Processing CreateTablet for tablet 16ab928a487c4f499aba3b366d01d89b (DEFAULT_TABLE table=default.Table_0d113826d96240549bc980febd8776e0 [id=f9582c45829d4f10af98e245dd41a907]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:32.562866 11752 ts_tablet_manager.cc:1918] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260926 07:35:32.564607 11314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c092aff8f2a041ccb09bc4b48e0f359c" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:32.566133 11546 leader_election.cc:302] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1
I20260926 07:35:32.564832 11752 log.cc:1196] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/05f071b4db7847a5adbd25a71cac3ea7
W20260926 07:35:32.567263 11546 leader_election.cc:341] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:32.567613 11546 leader_election.cc:302] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:32.569473 11752 ts_tablet_manager.cc:1939] T 05f071b4db7847a5adbd25a71cac3ea7 P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:32.557909 12168 ts_tablet_manager.cc:1918] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260926 07:35:32.572014 12168 log.cc:1196] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/05f071b4db7847a5adbd25a71cac3ea7
I20260926 07:35:32.572167 12125 raft_consensus.cc:2781] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:32.572475 12125 raft_consensus.cc:482] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:32.572772 12168 ts_tablet_manager.cc:1939] T 05f071b4db7847a5adbd25a71cac3ea7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:32.572926 12125 raft_consensus.cc:3037] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:32.558121 11313 raft_consensus.cc:3037] T 0eae53ce278749208492815b918582f3 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:32.576262 12168 tablet_replica.cc:330] T 640d21586bad4070957cff61bd174fd2 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:32.579591 11752 tablet_replica.cc:330] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:32.581331 12168 raft_consensus.cc:2229] T 640d21586bad4070957cff61bd174fd2 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:32.581881 12168 raft_consensus.cc:2258] T 640d21586bad4070957cff61bd174fd2 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:32.583815 12168 ts_tablet_manager.cc:1905] T 640d21586bad4070957cff61bd174fd2 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:32.584309 11290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31e135bc19994295a1fc1dcf39cca5cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:32.584252 11752 raft_consensus.cc:2229] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:32.588300 11752 pending_rounds.cc:62] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1: Trying to abort 1 pending ops.
I20260926 07:35:32.589358 11296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16ab928a487c4f499aba3b366d01d89b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:32.589685 11752 pending_rounds.cc:68] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333511707450859520 op_type: NO_OP noop_request { }
I20260926 07:35:32.548866 11871 ts_tablet_manager.cc:1905] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:32.591579 11752 raft_consensus.cc:2866] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 07:35:32.593189 11752 raft_consensus.cc:2258] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 07:35:32.598341 11401 consensus_peers.cc:487] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 55ad1d96e2314d1eb0a717a3e9ff31bc. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:32.602386 11752 ts_tablet_manager.cc:1905] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:32.605262 12200 raft_consensus.cc:2726] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:35:32.613652 11546 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.1:41829, user_credentials={real_user=slave}} blocked reactor thread for 46412.1us
I20260926 07:35:32.579537 12125 raft_consensus.cc:504] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.621469 11756 tablet_bootstrap.cc:492] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:32.621836 11756 ts_tablet_manager.cc:1397] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.475s user 0.054s sys 0.034s
I20260926 07:35:32.623394 12125 leader_election.cc:288] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:32.632577 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e28ca159d8349748812825ac1947c51" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
W20260926 07:35:32.636632 11545 leader_election.cc:341] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:32.638209 11313 raft_consensus.cc:2454] T 0eae53ce278749208492815b918582f3 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:32.624044 11756 raft_consensus.cc:348] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.644723 11756 raft_consensus.cc:374] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:32.644960 11756 raft_consensus.cc:729] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:32.645490 11756 consensus_queue.cc:260] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.655546 11756 ts_tablet_manager.cc:1428] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.033s user 0.005s sys 0.001s
I20260926 07:35:32.659304 11842 consensus_queue.cc:1035] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260926 07:35:32.661510 11546 consensus_peers.cc:487] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260926 07:35:32.662106 11546 consensus_peers.cc:487] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 034d508f0a164f3ea8308ea2d28818c4. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:32.489012 11117 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 4826318c83524f41840e770f33bd3696 (table default.Table_d6cdabd37ad94b339606911c37764fba [id=55a6e4b7598043d8bf49512f2542c589]). Expected version 4 got 3
I20260926 07:35:32.499197 11118 master_service.cc:946] ReplaceTablet: received request to replace tablet a0bdf6a4656e4217b0a7013fddc4cca8 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:32.581382 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet 05f071b4db7847a5adbd25a71cac3ea7 (table default.Table_b49978de02e64080bb6633a3b7c7f7ca [id=9e879069d99f41cdb271157af1a3fd1e]) successfully deleted
I20260926 07:35:32.590106 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet 05f071b4db7847a5adbd25a71cac3ea7 (table default.Table_b49978de02e64080bb6633a3b7c7f7ca [id=9e879069d99f41cdb271157af1a3fd1e]) successfully deleted
W20260926 07:35:32.664393 11106 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.171.1:41829, user_credentials={real_user=slave}} blocked reactor thread for 83031.7us
W20260926 07:35:32.664541 11105 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.171.2:42379, user_credentials={real_user=slave}} blocked reactor thread for 74461us
W20260926 07:35:32.665601 11105 catalog_manager.cc:4808] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 509f607b089c44bc9c3f80ed59a68c95 because the tablet was not found. No further retry: Not found: Tablet not found: 509f607b089c44bc9c3f80ed59a68c95
I20260926 07:35:32.666117 11117 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet ae16af9edff9401589bcf54eeb3e8dfe (table default.Table_f9178d5e3d934f1f9ef267a782316458 [id=0c18461963f444269787b667b3648709]). Expected version 2 got 0
I20260926 07:35:32.666451 11756 tablet_bootstrap.cc:492] T 4239f5d48512403bb711ece401f0756b P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:32.666854 11117 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 0eae53ce278749208492815b918582f3 (table default.Table_56493b3efdaa43e684afcd3d4b5dffdd [id=f66bffa952604b28bb51299de795ee2a]). Expected version 1 got 0
I20260926 07:35:32.667420 11117 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet a55e54d492214039b0294f954b5ef390 (table default.Table_72783eb75b984ed394613e1333b94227 [id=65c112d56f2a44cc9d840caadd218412]). Expected version 1 got 0
I20260926 07:35:32.667969 11117 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 0218d2a727f942e48d770878e6bf4a54 (table default.Table_e8b00b73be7c4818adf8d1e74d731ee6 [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b]). Expected version 3 got 0
I20260926 07:35:32.669044 11113 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet b862f325a9df49079066dc62fe1a61f1
I20260926 07:35:32.694783 11752 ts_tablet_manager.cc:1918] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:32.696961 11752 log.cc:1196] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/d6fc6178f1e54c3984a20873864233cd
I20260926 07:35:32.700762 11752 ts_tablet_manager.cc:1939] T d6fc6178f1e54c3984a20873864233cd P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:32.710716 11746 tablet_bootstrap.cc:492] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:32.711349 11746 ts_tablet_manager.cc:1397] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.283s user 0.010s sys 0.052s
I20260926 07:35:32.720541 12154 log.cc:824] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249: Log is configured to *not* fsync() on all Append() calls
I20260926 07:35:32.719802 11746 raft_consensus.cc:348] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.722805 11746 raft_consensus.cc:374] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:32.725968 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet d6fc6178f1e54c3984a20873864233cd (table default.Table_051aa34eea804c698486395b4bf2553f [id=f299d82b1a01434690a378d2494fd370]) successfully deleted
I20260926 07:35:32.726202 11752 tablet_replica.cc:330] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:32.727058 11752 raft_consensus.cc:2229] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:32.727684 11752 pending_rounds.cc:62] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1: Trying to abort 1 pending ops.
I20260926 07:35:32.727934 11752 pending_rounds.cc:68] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333511706593038336 op_type: NO_OP noop_request { }
I20260926 07:35:32.728304 11752 raft_consensus.cc:2866] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 07:35:32.728602 11752 raft_consensus.cc:2258] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:32.731670 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e28ca159d8349748812825ac1947c51" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:32.723775 11746 raft_consensus.cc:729] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:32.732236 11313 raft_consensus.cc:3037] T 3e28ca159d8349748812825ac1947c51 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:32.732735 11746 consensus_queue.cc:260] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.733721 11116 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 6d524be8509645a89a75b4bac15f0889
I20260926 07:35:32.735265 11116 master_service.cc:946] ReplaceTablet: received request to replace tablet d6fc6178f1e54c3984a20873864233cd from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:32.737169 11289 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d524be8509645a89a75b4bac15f0889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:32 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:32.738834 11440 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d524be8509645a89a75b4bac15f0889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:32 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:32.739389 11120 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_091d9aec1e664dc1aa7f5f1006b33eaf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:32.740818 11120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_091d9aec1e664dc1aa7f5f1006b33eaf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:32.744794 11515 heartbeater.cc:347] Connected to a master server at 127.10.171.62:37629
I20260926 07:35:32.749804 11746 ts_tablet_manager.cc:1428] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.038s user 0.003s sys 0.003s
I20260926 07:35:32.730198 11752 ts_tablet_manager.cc:1905] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:32.751793 11116 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_af06af7ab1b64367bfd384a03f121eda" } new_table_name: "default.Table_3de17a801f1c4fa99878e07e9df543f8" modify_external_catalogs: true
I20260926 07:35:32.753707 11842 raft_consensus.cc:482] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:32.754112 11842 raft_consensus.cc:504] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.755968 11842 leader_election.cc:288] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
W20260926 07:35:32.761806 11112 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5ee33141350b4c31aae79d2160e79b6f consensus state has no leader
I20260926 07:35:32.765831 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90f62593b6f844fc8a41b9be240101fd" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
W20260926 07:35:32.767977 11401 leader_election.cc:341] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:32.768989 11746 tablet_bootstrap.cc:492] T 04be706330924282961054c5835cec73 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:32.776832 11756 tablet_bootstrap.cc:654] T 4239f5d48512403bb711ece401f0756b P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:32.792932 11313 raft_consensus.cc:2454] T 3e28ca159d8349748812825ac1947c51 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6070e9e3d1234a9d83e49431416afade in term 1.
I20260926 07:35:32.809310 12154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 4 replicates
I20260926 07:35:32.812994 12154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249: Bootstrap complete.
I20260926 07:35:32.816303 11754 tablet_bootstrap.cc:492] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:32.816395 11746 tablet_bootstrap.cc:654] T 04be706330924282961054c5835cec73 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:32.816655 11754 ts_tablet_manager.cc:1397] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.485s user 0.014s sys 0.006s
I20260926 07:35:32.830521 11746 tablet_bootstrap.cc:492] T 04be706330924282961054c5835cec73 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:32.830921 11746 ts_tablet_manager.cc:1397] T 04be706330924282961054c5835cec73 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.062s user 0.007s sys 0.002s
I20260926 07:35:32.833120 11746 raft_consensus.cc:348] T 04be706330924282961054c5835cec73 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.833797 11746 raft_consensus.cc:374] T 04be706330924282961054c5835cec73 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:32.834038 11746 raft_consensus.cc:729] T 04be706330924282961054c5835cec73 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:32.813141 11582 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d524be8509645a89a75b4bac15f0889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:32 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:32.827960 11752 ts_tablet_manager.cc:1918] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:32.837226 11752 log.cc:1196] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/55ad1d96e2314d1eb0a717a3e9ff31bc
I20260926 07:35:32.838908 11746 consensus_queue.cc:260] T 04be706330924282961054c5835cec73 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:32.842734 11545 consensus_peers.cc:487] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade -> Peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Couldn't send request to peer 22f3902d01584936b20437606cb86dd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 034d508f0a164f3ea8308ea2d28818c4. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:32.843398 12202 consensus_queue.cc:1035] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:32.844190 11752 ts_tablet_manager.cc:1939] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:32.846393 11754 raft_consensus.cc:348] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:32.847024 11754 raft_consensus.cc:374] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:32.847316 11754 raft_consensus.cc:729] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:32.855916 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90f62593b6f844fc8a41b9be240101fd" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:32.856583 11609 raft_consensus.cc:2454] T 90f62593b6f844fc8a41b9be240101fd P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:32.857067 11746 ts_tablet_manager.cc:1428] T 04be706330924282961054c5835cec73 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.026s user 0.005s sys 0.000s
I20260926 07:35:32.857852 11746 tablet_bootstrap.cc:492] T 3e28ca159d8349748812825ac1947c51 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:32.857813 11402 leader_election.cc:302] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters: e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:32.859198 11842 raft_consensus.cc:2781] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:32.859498 11842 raft_consensus.cc:482] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:32.859905 11842 raft_consensus.cc:3037] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:32.860541 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet 55ad1d96e2314d1eb0a717a3e9ff31bc (table default.Table_4d463023afd047528c16b7ee4dabd36f [id=eeb221378f35425eb8d24b815f8a2181]) successfully deleted
W20260926 07:35:32.861263 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 6d54a5c32da5444ca0a2939461f3a0f6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d54a5c32da5444ca0a2939461f3a0f6 already in progress: creating tablet
W20260926 07:35:32.862142 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 6d524be8509645a89a75b4bac15f0889 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d524be8509645a89a75b4bac15f0889 already in progress: creating tablet
I20260926 07:35:32.848161 11754 consensus_queue.cc:260] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:32.873831 11754 ts_tablet_manager.cc:1428] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.057s user 0.005s sys 0.000s
I20260926 07:35:32.874246 11746 tablet_bootstrap.cc:654] T 3e28ca159d8349748812825ac1947c51 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:32.876930 12212 raft_consensus.cc:482] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:32.877353 12212 raft_consensus.cc:504] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.879266 11842 raft_consensus.cc:504] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.881757 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4239f5d48512403bb711ece401f0756b" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:32.896847 11609 raft_consensus.cc:2454] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:32.880242 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4239f5d48512403bb711ece401f0756b" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:32.859390 12168 ts_tablet_manager.cc:1918] T 640d21586bad4070957cff61bd174fd2 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:32.901191 11754 tablet_bootstrap.cc:492] T 0f20206ce6fd4fd395f1b2b0154245db P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:32.901160 12168 log.cc:1196] T 640d21586bad4070957cff61bd174fd2 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/640d21586bad4070957cff61bd174fd2
I20260926 07:35:32.901918 12168 ts_tablet_manager.cc:1939] T 640d21586bad4070957cff61bd174fd2 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:32.904925 11746 tablet_bootstrap.cc:492] T 3e28ca159d8349748812825ac1947c51 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:32.905545 11746 ts_tablet_manager.cc:1397] T 3e28ca159d8349748812825ac1947c51 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.048s user 0.011s sys 0.008s
I20260926 07:35:32.906004 11842 leader_election.cc:288] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
W20260926 07:35:32.698633 11546 leader_election.cc:341] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Not found: Tablet not found: afa901cdc6a84220bf36ee669a955b72
I20260926 07:35:32.925354 11546 leader_election.cc:302] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:32.904958 12168 tablet_replica.cc:330] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:32.928053 12202 raft_consensus.cc:2726] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:32.928745 12168 raft_consensus.cc:2229] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus shutting down.
W20260926 07:35:32.928891 11546 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.1:41829, user_credentials={real_user=slave}} blocked reactor thread for 266304us
I20260926 07:35:32.929260 12168 raft_consensus.cc:2258] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:32.931115 12168 ts_tablet_manager.cc:1905] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:32.928741 12154 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [term 1 FOLLOWER]: Replica starting. Triggering 4 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:32.898223 11402 leader_election.cc:302] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:32.907946 11746 raft_consensus.cc:348] T 3e28ca159d8349748812825ac1947c51 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.880795 12212 leader_election.cc:288] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:32.945605 11746 raft_consensus.cc:374] T 3e28ca159d8349748812825ac1947c51 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260926 07:35:32.945688 11401 leader_election.cc:341] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
W20260926 07:35:32.945719 11402 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.3:45127, user_credentials={real_user=slave}} blocked reactor thread for 47620us
I20260926 07:35:32.946014 11746 raft_consensus.cc:729] T 3e28ca159d8349748812825ac1947c51 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:32.946717 11746 consensus_queue.cc:260] T 3e28ca159d8349748812825ac1947c51 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.947430 12212 raft_consensus.cc:2781] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:32.947952 12212 raft_consensus.cc:482] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:32.948251 12212 raft_consensus.cc:3037] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:32.931510 12202 raft_consensus.cc:482] T dbd4f14abfea4799b75c48531412fc20 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:32.950685 11746 ts_tablet_manager.cc:1428] T 3e28ca159d8349748812825ac1947c51 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.045s user 0.007s sys 0.000s
I20260926 07:35:32.951028 12202 raft_consensus.cc:504] T dbd4f14abfea4799b75c48531412fc20 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:32.951838 11746 tablet_bootstrap.cc:492] T 9feaf1ed8ff34928b716fd0b4e41cdde P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:32.947194 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90f62593b6f844fc8a41b9be240101fd" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:32.952883 11609 raft_consensus.cc:3037] T 90f62593b6f844fc8a41b9be240101fd P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:32.952926 12202 leader_election.cc:288] T dbd4f14abfea4799b75c48531412fc20 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:32.954384 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd4f14abfea4799b75c48531412fc20" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
W20260926 07:35:32.956975 11545 leader_election.cc:341] T dbd4f14abfea4799b75c48531412fc20 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:32.958941 11746 tablet_bootstrap.cc:654] T 9feaf1ed8ff34928b716fd0b4e41cdde P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:32.959618 11609 raft_consensus.cc:2454] T 90f62593b6f844fc8a41b9be240101fd P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:32.961244 11402 leader_election.cc:302] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:32.962014 12214 raft_consensus.cc:2781] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:32.962509 12214 raft_consensus.cc:686] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:32.963323 12214 consensus_queue.cc:237] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.956435 12154 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6ef5b900d0f4e73a358f415efa97249, State: Initialized, Role: FOLLOWER
I20260926 07:35:32.972472 12154 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:32.979152 11871 ts_tablet_manager.cc:1918] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:32.979602 11871 log.cc:1196] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/acc5568dee9b473f8f5f1f8bc3b5504e
I20260926 07:35:32.980228 12212 raft_consensus.cc:504] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:32.984588 12202 consensus_queue.cc:1035] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:32.934473 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90f62593b6f844fc8a41b9be240101fd" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:32.958287 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd4f14abfea4799b75c48531412fc20" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:33.000586 12154 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:35:33.003147 11756 tablet_bootstrap.cc:492] T 4239f5d48512403bb711ece401f0756b P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:33.003571 11756 ts_tablet_manager.cc:1397] T 4239f5d48512403bb711ece401f0756b P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.339s user 0.015s sys 0.008s
I20260926 07:35:33.006543 12071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [term 1 FOLLOWER]: Advancing to term 3
I20260926 07:35:32.983534 11746 tablet_bootstrap.cc:492] T 9feaf1ed8ff34928b716fd0b4e41cdde P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:33.017804 11746 ts_tablet_manager.cc:1397] T 9feaf1ed8ff34928b716fd0b4e41cdde P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.066s user 0.010s sys 0.003s
I20260926 07:35:33.020682 11746 raft_consensus.cc:348] T 9feaf1ed8ff34928b716fd0b4e41cdde P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:33.020849 11756 raft_consensus.cc:348] T 4239f5d48512403bb711ece401f0756b P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:33.021476 11756 raft_consensus.cc:374] T 4239f5d48512403bb711ece401f0756b P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:33.021482 11746 raft_consensus.cc:374] T 9feaf1ed8ff34928b716fd0b4e41cdde P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:33.021711 11756 raft_consensus.cc:729] T 4239f5d48512403bb711ece401f0756b P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:33.021739 11746 raft_consensus.cc:729] T 9feaf1ed8ff34928b716fd0b4e41cdde P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:33.022444 12071 pending_rounds.cc:76] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249: Aborting all ops after (but not including) 5
I20260926 07:35:33.022248 11756 consensus_queue.cc:260] T 4239f5d48512403bb711ece401f0756b P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:33.022341 11746 consensus_queue.cc:260] T 9feaf1ed8ff34928b716fd0b4e41cdde P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:32.998245 11401 leader_election.cc:341] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:33.024880 12212 leader_election.cc:288] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:33.025794 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4239f5d48512403bb711ece401f0756b" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:33.027931 12071 pending_rounds.cc:97] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249: Aborting uncommitted WRITE_OP operation due to leader change: 1.6
I20260926 07:35:33.029654 12071 pending_rounds.cc:97] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249: Aborting uncommitted WRITE_OP operation due to leader change: 1.7
I20260926 07:35:33.030427 12071 pending_rounds.cc:97] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249: Aborting uncommitted WRITE_OP operation due to leader change: 1.8
I20260926 07:35:33.026410 11608 raft_consensus.cc:3037] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
W20260926 07:35:33.045843 11546 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.171.1:41829, user_credentials={real_user=slave}} blocked reactor thread for 61436.3us
I20260926 07:35:33.025170 11746 ts_tablet_manager.cc:1428] T 9feaf1ed8ff34928b716fd0b4e41cdde P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260926 07:35:33.053040 11746 tablet_bootstrap.cc:492] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:33.057250 11111 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_95dfccbf3504491bb70c84a9ac9038d1" } modify_external_catalogs: true
I20260926 07:35:33.057746 11111 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_95dfccbf3504491bb70c84a9ac9038d1" } modify_external_catalogs: true
I20260926 07:35:33.059078 11746 tablet_bootstrap.cc:654] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:33.046536 12214 raft_consensus.cc:482] T 04be706330924282961054c5835cec73 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:33.060387 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_7941154c519c4b23a2983c2b3ec82ec2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:33.060345 12214 raft_consensus.cc:504] T 04be706330924282961054c5835cec73 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:33.061749 11112 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7941154c519c4b23a2983c2b3ec82ec2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:33.066015 11756 ts_tablet_manager.cc:1428] T 4239f5d48512403bb711ece401f0756b P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.062s user 0.004s sys 0.002s
I20260926 07:35:33.068540 11756 tablet_bootstrap.cc:492] T c81a59361a6e42b7962b3e027c4a087f P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:33.071008 12214 leader_election.cc:288] T 04be706330924282961054c5835cec73 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:33.079612 11746 tablet_bootstrap.cc:492] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:33.080138 11746 ts_tablet_manager.cc:1397] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.027s user 0.006s sys 0.005s
I20260926 07:35:33.082633 11746 raft_consensus.cc:348] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:33.083549 11746 raft_consensus.cc:374] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:33.083823 11746 raft_consensus.cc:729] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:33.092741 11746 consensus_queue.cc:260] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:33.095824 11746 ts_tablet_manager.cc:1428] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.015s user 0.002s sys 0.003s
I20260926 07:35:33.096931 11746 tablet_bootstrap.cc:492] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:33.102552 11746 tablet_bootstrap.cc:654] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:33.102442 12168 ts_tablet_manager.cc:1918] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:33.108832 12168 log.cc:1196] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/d6fc6178f1e54c3984a20873864233cd
I20260926 07:35:33.102877 11756 tablet_bootstrap.cc:654] T c81a59361a6e42b7962b3e027c4a087f P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:33.113241 12168 ts_tablet_manager.cc:1939] T d6fc6178f1e54c3984a20873864233cd P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:33.065244 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04be706330924282961054c5835cec73" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:33.065587 11608 raft_consensus.cc:2454] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:33.117909 11402 leader_election.cc:302] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:33.118571 12214 raft_consensus.cc:2781] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:33.118839 11546 leader_election.cc:302] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1
I20260926 07:35:33.119796 12202 raft_consensus.cc:2781] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:33.120137 12125 consensus_queue.cc:1035] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:33.119045 12214 raft_consensus.cc:686] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
W20260926 07:35:33.121786 11546 leader_election.cc:341] T dbd4f14abfea4799b75c48531412fc20 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:33.122156 11546 leader_election.cc:302] T dbd4f14abfea4799b75c48531412fc20 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:33.122179 12214 consensus_queue.cc:237] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:33.122951 12125 raft_consensus.cc:2726] T dbd4f14abfea4799b75c48531412fc20 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:33.120328 12202 raft_consensus.cc:686] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Becoming Leader. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Running, Role: LEADER
I20260926 07:35:33.125214 12202 consensus_queue.cc:237] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:33.126186 11754 tablet_bootstrap.cc:654] T 0f20206ce6fd4fd395f1b2b0154245db P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:33.132943 11116 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6c42b518ef04429aa6b4e8f5deee05c8 consensus state has no leader
I20260926 07:35:33.135021 11118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet a0bdf6a4656e4217b0a7013fddc4cca8
I20260926 07:35:33.139147 11115 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet a55e54d492214039b0294f954b5ef390 (table default.Table_72783eb75b984ed394613e1333b94227 [id=65c112d56f2a44cc9d840caadd218412]). Expected version 1 got 0
I20260926 07:35:33.140096 11115 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet ae16af9edff9401589bcf54eeb3e8dfe (table default.Table_f9178d5e3d934f1f9ef267a782316458 [id=0c18461963f444269787b667b3648709]). Expected version 2 got 0
I20260926 07:35:33.141372 11440 tablet_service.cc:1514] Processing DeleteTablet for tablet a0bdf6a4656e4217b0a7013fddc4cca8 with delete_type TABLET_DATA_DELETED (replaced by tablet 089bac5feeaa4508bc863e446b172033) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:33.141848 11115 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 0218d2a727f942e48d770878e6bf4a54 (table default.Table_e8b00b73be7c4818adf8d1e74d731ee6 [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b]). Expected version 3 got 0
I20260926 07:35:33.142123 11752 tablet_replica.cc:330] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:33.143004 11752 raft_consensus.cc:2229] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:33.140648 11118 catalog_manager.cc:6565] ReplaceTablet: tablet a0bdf6a4656e4217b0a7013fddc4cca8 deleted and replaced by tablet 089bac5feeaa4508bc863e446b172033
I20260926 07:35:33.143436 11752 raft_consensus.cc:2258] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:33.144824 11590 tablet_service.cc:1514] Processing DeleteTablet for tablet a0bdf6a4656e4217b0a7013fddc4cca8 with delete_type TABLET_DATA_DELETED (replaced by tablet 089bac5feeaa4508bc863e446b172033) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:33.145272 11752 ts_tablet_manager.cc:1905] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:33.149917 11871 ts_tablet_manager.cc:1939] T acc5568dee9b473f8f5f1f8bc3b5504e P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:33.159435 12212 consensus_queue.cc:1035] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:33.178560 11116 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_53678f9c4c01403eb2d846d3242be0aa" } modify_external_catalogs: true
I20260926 07:35:33.179098 11116 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_53678f9c4c01403eb2d846d3242be0aa" } modify_external_catalogs: true
I20260926 07:35:33.176327 11752 ts_tablet_manager.cc:1918] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260926 07:35:33.181031 11752 log.cc:1196] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/a0bdf6a4656e4217b0a7013fddc4cca8
I20260926 07:35:33.181053 11118 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_e888a0ff22884ed0bf88475c3f95b42c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:33.182196 11752 ts_tablet_manager.cc:1939] T a0bdf6a4656e4217b0a7013fddc4cca8 P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
W20260926 07:35:33.182533 11118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e888a0ff22884ed0bf88475c3f95b42c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:33.183265 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet a0bdf6a4656e4217b0a7013fddc4cca8 (table default.Table_1da003049e4b46eb994aaa3f3094d63c [id=3ce63f9fb5ad4b82ba2fbbf2e3678271]) successfully deleted
I20260926 07:35:33.185516 11120 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_b90378c291a04ddda6fba25512c07fb4" } new_table_name: "default.Table_fd842e14c4984402b2b1b77baa1f9360" modify_external_catalogs: true
I20260926 07:35:33.201282 12214 raft_consensus.cc:482] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:33.201823 12214 raft_consensus.cc:504] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:33.205348 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c092aff8f2a041ccb09bc4b48e0f359c" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:33.205905 11609 raft_consensus.cc:2454] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:33.207463 11402 leader_election.cc:302] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:33.208204 12212 raft_consensus.cc:2781] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:33.208580 12212 raft_consensus.cc:482] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:33.210650 12212 raft_consensus.cc:3037] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:33.203879 12214 leader_election.cc:288] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:33.217021 12212 raft_consensus.cc:504] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:33.219161 11111 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 1654ed93f96a4b6391094adde0cd3d31
I20260926 07:35:33.220257 12212 leader_election.cc:288] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:33.280161 12237 raft_consensus.cc:482] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:33.283341 12237 raft_consensus.cc:504] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:33.287926 12237 leader_election.cc:288] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
W20260926 07:35:33.289747 11104 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet f31a069dc888487ebbbb1cb91e886ad0 (table default.Table_ba08930510174e65840452201a15f719 [id=47d4200172794cc78a6f386fee4b15b4], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet f31a069dc888487ebbbb1cb91e886ad0 consensus state has no leader
I20260926 07:35:33.289942 12237 consensus_queue.cc:1035] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 07:35:33.300855 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9feaf1ed8ff34928b716fd0b4e41cdde" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:33.302986 11463 raft_consensus.cc:2454] T 9feaf1ed8ff34928b716fd0b4e41cdde P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 0.
I20260926 07:35:33.288913 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_f0a09bb101f848819059089fd6a2aa65"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:33.304899 11112 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f0a09bb101f848819059089fd6a2aa65 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:33.315941 11247 leader_election.cc:302] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:33.316903 12237 raft_consensus.cc:2781] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:33.317224 12237 raft_consensus.cc:482] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:33.317538 12237 raft_consensus.cc:3037] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:33.325979 12168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 138) took 1334 ms. Trace:
I20260926 07:35:33.326320 12168 rpcz_store.cc:276] 0926 07:35:31.991853 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:31.992512 (+ 659us) service_pool.cc:227] Handling call
0926 07:35:32.904804 (+912292us) ts_tablet_manager.cc:1145] Deleting tablet d6fc6178f1e54c3984a20873864233cd
0926 07:35:32.904849 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:33.050752 (+145903us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:33.102154 (+ 51402us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:33.325428 (+223274us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b5400141254. stack: 00007fedc0a39033 00007fedc0a38f30 00007fedc05ba534 00000000004db2b4 00000000004db249 00007fedc7945471 00007fedc79c82d4 00007fedc79b51b1 00007fedc7dde9c1 00007fedc89ebe32 00007fedc8a1be10 00007fedc8a02e0d 00007fedc8a02db9 00007fedc8a02d49 00007fedc8a02d11 00007fedc8a01f3d
0926 07:35:33.325796 (+ 368us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":480625920,"tablet-delete.queue_time_us":911786}
I20260926 07:35:33.329209 12168 tablet_replica.cc:330] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:33.330081 12168 raft_consensus.cc:2229] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:33.330664 12168 pending_rounds.cc:62] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e: Trying to abort 1 pending ops.
I20260926 07:35:33.330981 12168 pending_rounds.cc:68] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333511707301326848 op_type: NO_OP noop_request { }
I20260926 07:35:33.331560 12168 raft_consensus.cc:2866] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260926 07:35:33.332043 12168 raft_consensus.cc:2258] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:33.339035 11118 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_a2d61651f47c4fd599f740faeec0f7d3" } new_table_name: "default.Table_c2bf1974cc4f40659fbabf743e08210f" modify_external_catalogs: true
I20260926 07:35:33.340509 11756 tablet_bootstrap.cc:492] T c81a59361a6e42b7962b3e027c4a087f P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:33.341001 11756 ts_tablet_manager.cc:1397] T c81a59361a6e42b7962b3e027c4a087f P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.274s user 0.011s sys 0.003s
I20260926 07:35:33.342619 11842 raft_consensus.cc:482] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:33.342679 12237 raft_consensus.cc:504] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:33.343048 11842 raft_consensus.cc:504] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:33.343703 11756 raft_consensus.cc:348] T c81a59361a6e42b7962b3e027c4a087f P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:33.344883 11842 leader_election.cc:288] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:33.344367 11756 raft_consensus.cc:374] T c81a59361a6e42b7962b3e027c4a087f P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:33.349865 11756 raft_consensus.cc:729] T c81a59361a6e42b7962b3e027c4a087f P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:33.350474 11756 consensus_queue.cc:260] T c81a59361a6e42b7962b3e027c4a087f P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:33.352885 11756 ts_tablet_manager.cc:1428] T c81a59361a6e42b7962b3e027c4a087f P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260926 07:35:33.353806 11756 tablet_bootstrap.cc:492] T 90f62593b6f844fc8a41b9be240101fd P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:33.359194 11756 tablet_bootstrap.cc:654] T 90f62593b6f844fc8a41b9be240101fd P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:33.362087 12125 raft_consensus.cc:482] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:33.362538 12125 raft_consensus.cc:504] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
W20260926 07:35:33.366798 11118 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dca3e232caf14ee88adf61d440113262 has no consensus state
I20260926 07:35:33.368206 11118 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_597d2bba0c2b4bfc8c1e14e454b61b86"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:33.369206 11462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9feaf1ed8ff34928b716fd0b4e41cdde" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1"
I20260926 07:35:33.370476 11462 raft_consensus.cc:3037] T 9feaf1ed8ff34928b716fd0b4e41cdde P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
W20260926 07:35:33.374200 11118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_597d2bba0c2b4bfc8c1e14e454b61b86 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:33.380417 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c092aff8f2a041ccb09bc4b48e0f359c" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:33.380935 11609 raft_consensus.cc:3037] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:33.384083 11611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04180ace5d4548dfbcd226bf229d5c26" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:33.386147 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 121) took 3723 ms. Trace:
I20260926 07:35:33.386457 11871 rpcz_store.cc:276] 0926 07:35:29.662893 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.662994 (+ 101us) service_pool.cc:227] Handling call
0926 07:35:32.251043 (+2588049us) ts_tablet_manager.cc:1145] Deleting tablet acc5568dee9b473f8f5f1f8bc3b5504e
0926 07:35:32.251081 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:32.599295 (+348214us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:32.978943 (+379648us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:33.386093 (+407150us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":286539,"tablet-delete.queue_time_us":2587519}
I20260926 07:35:33.387650 11871 tablet_replica.cc:330] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
W20260926 07:35:33.388813 11545 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.171.2:42379, user_credentials={real_user=slave}} blocked reactor thread for 161005us
I20260926 07:35:33.391183 11611 raft_consensus.cc:2454] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:33.392959 11871 raft_consensus.cc:2229] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:33.392876 11402 leader_election.cc:302] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:33.393373 11871 raft_consensus.cc:2258] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:33.394932 11871 ts_tablet_manager.cc:1905] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:33.397044 11842 raft_consensus.cc:2781] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:33.397354 11842 raft_consensus.cc:482] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:33.397670 11842 raft_consensus.cc:3037] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:33.398411 12125 leader_election.cc:288] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:33.400892 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cd482fb159d418291cb02bc9138b076" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
W20260926 07:35:33.402029 11545 leader_election.cc:341] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:33.405045 11754 tablet_bootstrap.cc:492] T 0f20206ce6fd4fd395f1b2b0154245db P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:33.405488 11754 ts_tablet_manager.cc:1397] T 0f20206ce6fd4fd395f1b2b0154245db P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.504s user 0.007s sys 0.004s
I20260926 07:35:33.408424 11465 raft_consensus.cc:3037] T 04be706330924282961054c5835cec73 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:33.417834 11754 raft_consensus.cc:348] T 0f20206ce6fd4fd395f1b2b0154245db P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:33.418470 11754 raft_consensus.cc:374] T 0f20206ce6fd4fd395f1b2b0154245db P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:33.418697 11754 raft_consensus.cc:729] T 0f20206ce6fd4fd395f1b2b0154245db P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
W20260926 07:35:33.429368 11546 consensus_peers.cc:487] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:33.437152 11660 heartbeater.cc:503] Master 127.10.171.62:37629 was elected leader, sending a full tablet report...
I20260926 07:35:33.390687 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet acc5568dee9b473f8f5f1f8bc3b5504e (table default.Table_23c2afadc1164dfab1f47d7c0b373e5f [id=6806510fcf2d4b17960903772e08d49b]) successfully deleted
W20260926 07:35:33.441792 11107 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.171.3:45127, user_credentials={real_user=slave}} blocked reactor thread for 51128.1us
I20260926 07:35:33.443547 11113 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet ae16af9edff9401589bcf54eeb3e8dfe (table default.Table_f9178d5e3d934f1f9ef267a782316458 [id=0c18461963f444269787b667b3648709]). Expected version 2 got 0
I20260926 07:35:33.445114 11113 catalog_manager.cc:5495] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e reported cstate change: term changed from 0 to 1, leader changed from <none> to e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1). New cstate: current_term: 1 leader_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } health_report { overall_health: UNKNOWN } } }
I20260926 07:35:33.446343 11113 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet 0218d2a727f942e48d770878e6bf4a54 (table default.Table_e8b00b73be7c4818adf8d1e74d731ee6 [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b]). Expected version 3 got 0
I20260926 07:35:33.451260 11440 tablet_service.cc:1467] Processing CreateTablet for tablet dca3e232caf14ee88adf61d440113262 (DEFAULT_TABLE table=default.Table_c2bf1974cc4f40659fbabf743e08210f [id=cd8850cf63a54cb2ba7f33fd7fe20aa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:33.446450 11842 raft_consensus.cc:504] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:33.455505 11112 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_d0f2927c94ad48919b98e4f4199b42cd" } modify_external_catalogs: true
I20260926 07:35:33.451629 11590 tablet_service.cc:1467] Processing CreateTablet for tablet dca3e232caf14ee88adf61d440113262 (DEFAULT_TABLE table=default.Table_c2bf1974cc4f40659fbabf743e08210f [id=cd8850cf63a54cb2ba7f33fd7fe20aa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:33.458792 11590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dca3e232caf14ee88adf61d440113262. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:33.454542 11609 raft_consensus.cc:2454] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:33.468636 11402 leader_election.cc:302] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:33.469517 12212 raft_consensus.cc:2781] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:33.455622 11754 consensus_queue.cc:260] T 0f20206ce6fd4fd395f1b2b0154245db P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:33.469972 12212 raft_consensus.cc:686] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:33.470857 12212 consensus_queue.cc:237] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:33.471853 11754 ts_tablet_manager.cc:1428] T 0f20206ce6fd4fd395f1b2b0154245db P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.064s user 0.004s sys 0.000s
I20260926 07:35:33.472616 11754 tablet_bootstrap.cc:492] T 17871fbd15064daba6e80a34a1b221d3 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:33.476042 11112 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_d0f2927c94ad48919b98e4f4199b42cd" } modify_external_catalogs: true
I20260926 07:35:33.476905 11445 tablet_service.cc:1467] Processing CreateTablet for tablet 176db565ef8f4260a89b3eaa7d561671 (DEFAULT_TABLE table=default.Table_adafc8d10ea04a3ea42e2766739e9451 [id=11cde589c6d94849bd1448aca937062b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:33.477348 11583 tablet_service.cc:1467] Processing CreateTablet for tablet 176db565ef8f4260a89b3eaa7d561671 (DEFAULT_TABLE table=default.Table_adafc8d10ea04a3ea42e2766739e9451 [id=11cde589c6d94849bd1448aca937062b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:33.478236 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 176db565ef8f4260a89b3eaa7d561671. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:33.466708 11113 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet 0eae53ce278749208492815b918582f3 (table default.Table_56493b3efdaa43e684afcd3d4b5dffdd [id=f66bffa952604b28bb51299de795ee2a]). Expected version 1 got 0
I20260926 07:35:33.483529 11842 leader_election.cc:288] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:33.484496 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04180ace5d4548dfbcd226bf229d5c26" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:33.485174 11609 raft_consensus.cc:3037] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:33.505640 11609 raft_consensus.cc:2454] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:33.527370 11402 leader_election.cc:302] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:33.538308 12212 raft_consensus.cc:2781] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:33.538805 12212 raft_consensus.cc:686] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:33.539256 11444 tablet_service.cc:1467] Processing CreateTablet for tablet 246c13d624df4959b7b9e6e811da6818 (DEFAULT_TABLE table=default.Table_676572a0e9ac48f5a9ead303605b7ffa [id=2e98f0cd8cd94915811e5c6240bd897d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:33.539588 12212 consensus_queue.cc:237] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:33.557155 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9feaf1ed8ff34928b716fd0b4e41cdde" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:33.557626 11118 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_1887d867f46048ed8cdb47823c34d138"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:33.566883 11440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dca3e232caf14ee88adf61d440113262. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:33.568550 11445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 176db565ef8f4260a89b3eaa7d561671. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:33.583449 11119 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet ae16af9edff9401589bcf54eeb3e8dfe (table default.Table_f9178d5e3d934f1f9ef267a782316458 [id=0c18461963f444269787b667b3648709]). Expected version 2 got 0
I20260926 07:35:33.592741 11119 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 0218d2a727f942e48d770878e6bf4a54 (table default.Table_e8b00b73be7c4818adf8d1e74d731ee6 [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b]). Expected version 3 got 0
I20260926 07:35:33.593453 11119 catalog_manager.cc:5495] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22f3902d01584936b20437606cb86dd1 (127.10.171.2). New cstate: current_term: 1 leader_uuid: "22f3902d01584936b20437606cb86dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } health_report { overall_health: UNKNOWN } } }
I20260926 07:35:33.594668 11119 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 0eae53ce278749208492815b918582f3 (table default.Table_56493b3efdaa43e684afcd3d4b5dffdd [id=f66bffa952604b28bb51299de795ee2a]). Expected version 1 got 0
W20260926 07:35:33.597160 11118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1887d867f46048ed8cdb47823c34d138 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:33.598811 11444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 246c13d624df4959b7b9e6e811da6818. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:33.599085 11462 raft_consensus.cc:2454] T 9feaf1ed8ff34928b716fd0b4e41cdde P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bfebb34fa14cd2ad4843fffbe5c54e in term 1.
I20260926 07:35:33.591500 11112 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 7019af1230224e999e078b21ea5a30a5
I20260926 07:35:33.606531 11582 tablet_service.cc:1514] Processing DeleteTablet for tablet 7019af1230224e999e078b21ea5a30a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:33 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:33.637131 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9feaf1ed8ff34928b716fd0b4e41cdde" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:33.637247 11111 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 0eae53ce278749208492815b918582f3 (table default.Table_56493b3efdaa43e684afcd3d4b5dffdd [id=f66bffa952604b28bb51299de795ee2a]). Expected version 1 got 0
I20260926 07:35:33.612692 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_bd94d128c83c44a3bcc621acb91820c0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:33.642513 11112 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd94d128c83c44a3bcc621acb91820c0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:33.646205 11111 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet ae16af9edff9401589bcf54eeb3e8dfe (table default.Table_f9178d5e3d934f1f9ef267a782316458 [id=0c18461963f444269787b667b3648709]). Expected version 2 got 0
I20260926 07:35:33.646793 11111 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 0218d2a727f942e48d770878e6bf4a54 (table default.Table_e8b00b73be7c4818adf8d1e74d731ee6 [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b]). Expected version 3 got 0
I20260926 07:35:33.667124 11114 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: default.Table_1da003049e4b46eb994aaa3f3094d63c [id=3ce63f9fb5ad4b82ba2fbbf2e3678271] alter complete (version 2)
I20260926 07:35:33.678086 11582 tablet_service.cc:1467] Processing CreateTablet for tablet 6acc4f70e461477a97d53136e14e4961 (DEFAULT_TABLE table=default.Table_19ac6fcbe1b34c2d9f649f43ab3bdffd [id=029de7690a234fc489211d8a0f40043e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:33.679164 11582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6acc4f70e461477a97d53136e14e4961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:33.681998 11591 tablet_service.cc:1467] Processing CreateTablet for tablet 671d5729b087443fac769b2db41b52aa (DEFAULT_TABLE table=default.Table_f3f760b496a54e37b41668ee7a9585d5 [id=4a0d3a23a87e45a894c90d5f136dc362]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:33.682996 11591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 671d5729b087443fac769b2db41b52aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:33.684815 11444 tablet_service.cc:1467] Processing CreateTablet for tablet 6acc4f70e461477a97d53136e14e4961 (DEFAULT_TABLE table=default.Table_19ac6fcbe1b34c2d9f649f43ab3bdffd [id=029de7690a234fc489211d8a0f40043e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 07:35:33.686077 12154 thread.cc:636] rpc reactor (reactor) Time spent creating pthread: real 0.678s user 0.064s sys 0.091s
W20260926 07:35:33.686391 12154 thread.cc:602] rpc reactor (reactor) Time spent starting thread: real 0.679s user 0.064s sys 0.091s
I20260926 07:35:33.686249 11439 tablet_service.cc:1467] Processing CreateTablet for tablet 671d5729b087443fac769b2db41b52aa (DEFAULT_TABLE table=default.Table_f3f760b496a54e37b41668ee7a9585d5 [id=4a0d3a23a87e45a894c90d5f136dc362]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:33.689002 11444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6acc4f70e461477a97d53136e14e4961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:33.689694 11439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 671d5729b087443fac769b2db41b52aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:33.699698 11118 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_4d993e44ddc44452a2be1c319f81f9e8" } new_table_name: "default.Table_034a885a3ed648e6a5df65b50925a7cd" modify_external_catalogs: true
I20260926 07:35:33.702807 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet d6fc6178f1e54c3984a20873864233cd (table default.Table_051aa34eea804c698486395b4bf2553f [id=f299d82b1a01434690a378d2494fd370]) successfully deleted
W20260926 07:35:33.718432 11120 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2dd5175f0ca941f8b3856ea415e2ee68 consensus state has no leader
I20260926 07:35:33.719408 11120 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_d6bf3b3f4b934ec093561293afadaea1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:33.720762 11120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d6bf3b3f4b934ec093561293afadaea1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:33.724728 11116 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 4088ccb81e844cc29e1735bb341e4a92
I20260926 07:35:33.726867 11116 master_service.cc:946] ReplaceTablet: received request to replace tablet c092aff8f2a041ccb09bc4b48e0f359c from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:33.729449 11581 tablet_service.cc:1514] Processing DeleteTablet for tablet 4088ccb81e844cc29e1735bb341e4a92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:33 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:33.733825 11590 tablet_service.cc:1467] Processing CreateTablet for tablet 246c13d624df4959b7b9e6e811da6818 (DEFAULT_TABLE table=default.Table_676572a0e9ac48f5a9ead303605b7ffa [id=2e98f0cd8cd94915811e5c6240bd897d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:33.734851 11590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 246c13d624df4959b7b9e6e811da6818. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:33.734987 11114 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.1:34359 (request call id 44) took 2244 ms (client timeout 1000 ms). Trace:
W20260926 07:35:33.735316 11114 rpcz_store.cc:269] 0926 07:35:31.490704 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:31.585749 (+ 95045us) service_pool.cc:227] Handling call
0926 07:35:33.734906 (+2149157us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:31.921244 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:31.921580 (+ 336us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:31.921601 (+ 21us) write_op.cc:435] Acquired schema lock
0926 07:35:31.921613 (+ 12us) tablet.cc:660] Decoding operations
0926 07:35:31.921833 (+ 220us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:31.921863 (+ 30us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:31.921883 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0926 07:35:31.922008 (+ 125us) tablet.cc:699] Row locks acquired
0926 07:35:31.922022 (+ 14us) write_op.cc:260] PREPARE: finished
0926 07:35:31.922110 (+ 88us) write_op.cc:270] Start()
0926 07:35:31.922180 (+ 70us) write_op.cc:276] Timestamp: P: 1790408131922068 usec, L: 0
0926 07:35:31.922195 (+ 15us) op_driver.cc:331] REPLICATION: starting
0926 07:35:31.922635 (+ 440us) log.cc:842] Serialized 530 byte log entry
0926 07:35:32.052056 (+129421us) op_driver.cc:444] REPLICATION: finished
0926 07:35:32.053986 (+ 1930us) write_op.cc:301] APPLY: starting
0926 07:35:32.054042 (+ 56us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:32.054126 (+ 84us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:32.054257 (+ 131us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:32.054413 (+ 156us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:32.054539 (+ 126us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:32.054578 (+ 39us) write_op.cc:312] APPLY: finished
0926 07:35:32.054779 (+ 201us) log.cc:842] Serialized 23 byte log entry
0926 07:35:32.054942 (+ 163us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:32.054997 (+ 55us) write_op.cc:454] Released schema lock
0926 07:35:32.055023 (+ 26us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:32.055195 (+ 172us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:32.055220 (+ 25us) write_op.cc:454] Released schema lock
Related trace 'op':
0926 07:35:32.058245 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:32.058571 (+ 326us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:32.058593 (+ 22us) write_op.cc:435] Acquired schema lock
0926 07:35:32.058605 (+ 12us) tablet.cc:660] Decoding operations
0926 07:35:32.058826 (+ 221us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:32.058857 (+ 31us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:32.058880 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0926 07:35:32.059008 (+ 128us) tablet.cc:699] Row locks acquired
0926 07:35:32.059021 (+ 13us) write_op.cc:260] PREPARE: finished
0926 07:35:32.059119 (+ 98us) write_op.cc:270] Start()
0926 07:35:32.059195 (+ 76us) write_op.cc:276] Timestamp: P: 1790408132059063 usec, L: 0
0926 07:35:32.059213 (+ 18us) op_driver.cc:331] REPLICATION: starting
0926 07:35:32.059684 (+ 471us) log.cc:842] Serialized 356 byte log entry
0926 07:35:32.134460 (+ 74776us) op_driver.cc:444] REPLICATION: finished
0926 07:35:32.134938 (+ 478us) write_op.cc:301] APPLY: starting
0926 07:35:32.134987 (+ 49us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:32.135065 (+ 78us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:32.135149 (+ 84us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:32.135280 (+ 131us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:32.135399 (+ 119us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:32.135444 (+ 45us) write_op.cc:312] APPLY: finished
0926 07:35:32.135583 (+ 139us) log.cc:842] Serialized 23 byte log entry
0926 07:35:32.135885 (+ 302us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:32.135960 (+ 75us) write_op.cc:454] Released schema lock
0926 07:35:32.135990 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:32.136241 (+ 251us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:32.136265 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":1768,"apply.run_cpu_time_us":1396,"apply.run_wall_time_us":1395,"mutex_wait_us":36447,"num_ops":1,"prepare.queue_time_us":44064,"prepare.run_cpu_time_us":1615,"prepare.run_wall_time_us":26466,"replication_time_us":129785}],["op",{"apply.queue_time_us":338,"apply.run_cpu_time_us":1303,"apply.run_wall_time_us":1518,"num_ops":1,"prepare.queue_time_us":371,"prepare.run_cpu_time_us":1632,"prepare.run_wall_time_us":3684,"replication_time_us":75146}]]}
I20260926 07:35:33.742933 12212 raft_consensus.cc:482] T 3e28ca159d8349748812825ac1947c51 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:33.743391 12212 raft_consensus.cc:504] T 3e28ca159d8349748812825ac1947c51 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:33.752247 12125 raft_consensus.cc:482] T 0f20206ce6fd4fd395f1b2b0154245db P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:33.752763 12125 raft_consensus.cc:504] T 0f20206ce6fd4fd395f1b2b0154245db P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:33.753336 11754 tablet_bootstrap.cc:654] T 17871fbd15064daba6e80a34a1b221d3 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:33.755100 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e28ca159d8349748812825ac1947c51" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:33.755338 11462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f20206ce6fd4fd395f1b2b0154245db" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
W20260926 07:35:33.756763 11545 leader_election.cc:341] T 0f20206ce6fd4fd395f1b2b0154245db P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:33.756865 12212 leader_election.cc:288] T 3e28ca159d8349748812825ac1947c51 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:33.361052 12168 ts_tablet_manager.cc:1905] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:33.369077 12237 leader_election.cc:288] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Requested vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:33.423946 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cd482fb159d418291cb02bc9138b076" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:33.608289 11247 leader_election.cc:302] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e; no voters:
I20260926 07:35:33.704939 11289 tablet_service.cc:1514] Processing DeleteTablet for tablet a0bdf6a4656e4217b0a7013fddc4cca8 with delete_type TABLET_DATA_DELETED (replaced by tablet 089bac5feeaa4508bc863e446b172033) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:33.706192 11290 tablet_service.cc:1467] Processing CreateTablet for tablet dca3e232caf14ee88adf61d440113262 (DEFAULT_TABLE table=default.Table_c2bf1974cc4f40659fbabf743e08210f [id=cd8850cf63a54cb2ba7f33fd7fe20aa8]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 07:35:33.761197 11247 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.2:42379, user_credentials={real_user=slave}} blocked reactor thread for 152982us
I20260926 07:35:33.706323 11296 tablet_service.cc:1467] Processing CreateTablet for tablet 176db565ef8f4260a89b3eaa7d561671 (DEFAULT_TABLE table=default.Table_adafc8d10ea04a3ea42e2766739e9451 [id=11cde589c6d94849bd1448aca937062b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:33.707545 11288 tablet_service.cc:1467] Processing CreateTablet for tablet 246c13d624df4959b7b9e6e811da6818 (DEFAULT_TABLE table=default.Table_676572a0e9ac48f5a9ead303605b7ffa [id=2e98f0cd8cd94915811e5c6240bd897d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:33.707792 11292 tablet_service.cc:1467] Processing CreateTablet for tablet 6acc4f70e461477a97d53136e14e4961 (DEFAULT_TABLE table=default.Table_19ac6fcbe1b34c2d9f649f43ab3bdffd [id=029de7690a234fc489211d8a0f40043e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:33.708323 11286 tablet_service.cc:1514] Processing DeleteTablet for tablet 7019af1230224e999e078b21ea5a30a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:33 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:33.708796 11314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4239f5d48512403bb711ece401f0756b" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:33.763564 11314 raft_consensus.cc:3037] T 4239f5d48512403bb711ece401f0756b P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:33.709513 11285 tablet_service.cc:1467] Processing CreateTablet for tablet 671d5729b087443fac769b2db41b52aa (DEFAULT_TABLE table=default.Table_f3f760b496a54e37b41668ee7a9585d5 [id=4a0d3a23a87e45a894c90d5f136dc362]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:33.767515 12237 raft_consensus.cc:2781] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:33.767966 12237 raft_consensus.cc:686] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Becoming Leader. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Running, Role: LEADER
I20260926 07:35:33.768733 12237 consensus_queue.cc:237] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:33.786315 11546 leader_election.cc:341] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:33.789750 11546 leader_election.cc:302] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:33.792323 12133 raft_consensus.cc:2726] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:33.797135 11445 tablet_service.cc:1514] Processing DeleteTablet for tablet 7019af1230224e999e078b21ea5a30a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:33 UTC) from {username='slave'} at 127.0.0.1:40244
W20260926 07:35:33.721282 11249 leader_election.cc:341] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: Tablet not RUNNING: STOPPED
W20260926 07:35:33.809482 11746 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.704s user 0.083s sys 0.118s
W20260926 07:35:33.809751 11746 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.705s user 0.083s sys 0.118s
I20260926 07:35:33.821058 11746 tablet_bootstrap.cc:492] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:33.821436 11746 ts_tablet_manager.cc:1397] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.725s user 0.089s sys 0.121s
I20260926 07:35:33.823441 11746 raft_consensus.cc:348] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:33.832415 11583 tablet_service.cc:1467] Processing CreateTablet for tablet 4d0ac73eb239499083eead51e0a7e229 (DEFAULT_TABLE table=default.Table_e0f192a8a56f4343afa3cb8b77b6148d [id=f47c787764724aec9b36787e78b256bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:33.837566 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d0ac73eb239499083eead51e0a7e229. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:33.725557 12253 tablet.cc:1722] T ae16af9edff9401589bcf54eeb3e8dfe P e9bfebb34fa14cd2ad4843fffbe5c54e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 07:35:33.832588 11746 raft_consensus.cc:374] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:33.840420 11746 raft_consensus.cc:729] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:33.840999 11746 consensus_queue.cc:260] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
W20260926 07:35:33.805755 11249 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.3:45127, user_credentials={real_user=slave}} blocked reactor thread for 169137us
W20260926 07:35:33.849318 11249 leader_election.cc:341] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Illegal state: Tablet not RUNNING: STOPPED
I20260926 07:35:33.850960 11440 tablet_service.cc:1467] Processing CreateTablet for tablet 4d0ac73eb239499083eead51e0a7e229 (DEFAULT_TABLE table=default.Table_e0f192a8a56f4343afa3cb8b77b6148d [id=f47c787764724aec9b36787e78b256bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:33.851905 11746 ts_tablet_manager.cc:1428] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20260926 07:35:33.852033 12125 leader_election.cc:288] T 0f20206ce6fd4fd395f1b2b0154245db P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:33.852694 11440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d0ac73eb239499083eead51e0a7e229. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:33.853153 11746 tablet_bootstrap.cc:492] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:33.853624 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f20206ce6fd4fd395f1b2b0154245db" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
W20260926 07:35:33.855173 11546 leader_election.cc:341] T 0f20206ce6fd4fd395f1b2b0154245db P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:33.855647 11546 leader_election.cc:302] T 0f20206ce6fd4fd395f1b2b0154245db P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:33.811887 11112 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_c19ab6c510e64f6bb846aa3dc6f1c2fe" } modify_external_catalogs: true
I20260926 07:35:33.859515 11112 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_c19ab6c510e64f6bb846aa3dc6f1c2fe" } modify_external_catalogs: true
W20260926 07:35:33.864034 11105 catalog_manager.cc:4808] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 7019af1230224e999e078b21ea5a30a5 because the tablet was not found. No further retry: Not found: Tablet not found: 7019af1230224e999e078b21ea5a30a5
I20260926 07:35:33.868508 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04be706330924282961054c5835cec73" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:33.871172 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c092aff8f2a041ccb09bc4b48e0f359c" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:33.872471 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c092aff8f2a041ccb09bc4b48e0f359c" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:33.875159 11444 tablet_service.cc:1514] Processing DeleteTablet for tablet 4088ccb81e844cc29e1735bb341e4a92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:33 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:33.878839 11746 tablet_bootstrap.cc:654] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:33.882328 12133 raft_consensus.cc:2726] T 0f20206ce6fd4fd395f1b2b0154245db P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:33.884287 11581 tablet_service.cc:1467] Processing CreateTablet for tablet 3f3986a19a5044159c89ae6ba72b247c (DEFAULT_TABLE table=default.Table_53f51dc492ee41569d26cfe526418f7b [id=c4422fedd4854d4d94a16d9fb3311e53]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 07:35:33.886104 11105 catalog_manager.cc:4808] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 4088ccb81e844cc29e1735bb341e4a92 because the tablet was not found. No further retry: Not found: Tablet not found: 4088ccb81e844cc29e1735bb341e4a92
I20260926 07:35:33.890779 11290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dca3e232caf14ee88adf61d440113262. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:33.891311 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04180ace5d4548dfbcd226bf229d5c26" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:33.892468 11296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 176db565ef8f4260a89b3eaa7d561671. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:33.894043 11288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 246c13d624df4959b7b9e6e811da6818. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:33.895929 11746 tablet_bootstrap.cc:492] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:33.896080 11285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 671d5729b087443fac769b2db41b52aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:33.897118 11746 ts_tablet_manager.cc:1397] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.044s user 0.010s sys 0.000s
I20260926 07:35:33.897598 11292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6acc4f70e461477a97d53136e14e4961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:33.904369 12212 consensus_queue.cc:1035] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:33.920490 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04180ace5d4548dfbcd226bf229d5c26" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:33.929939 11871 ts_tablet_manager.cc:1918] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:33.930348 11871 log.cc:1196] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/9feaf1ed8ff34928b716fd0b4e41cdde
I20260926 07:35:33.940692 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e28ca159d8349748812825ac1947c51" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:33.941458 11313 raft_consensus.cc:2379] T 3e28ca159d8349748812825ac1947c51 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 22f3902d01584936b20437606cb86dd1 in current term 1: Already voted for candidate 6070e9e3d1234a9d83e49431416afade in this term.
I20260926 07:35:33.905575 11444 tablet_service.cc:1467] Processing CreateTablet for tablet 3f3986a19a5044159c89ae6ba72b247c (DEFAULT_TABLE table=default.Table_53f51dc492ee41569d26cfe526418f7b [id=c4422fedd4854d4d94a16d9fb3311e53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:33.943071 11444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f3986a19a5044159c89ae6ba72b247c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:33.909178 11746 raft_consensus.cc:348] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:33.943949 11746 raft_consensus.cc:374] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:33.944207 11746 raft_consensus.cc:729] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:33.944813 11746 consensus_queue.cc:260] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
W20260926 07:35:33.947813 11249 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.3:45127, user_credentials={real_user=slave}} blocked reactor thread for 98807us
I20260926 07:35:33.949996 12237 consensus_queue.cc:1035] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:33.955096 11746 ts_tablet_manager.cc:1428] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.058s user 0.005s sys 0.000s
I20260926 07:35:33.955977 11746 tablet_bootstrap.cc:492] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
W20260926 07:35:33.958549 11401 leader_election.cc:341] T 04be706330924282961054c5835cec73 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: Tablet not RUNNING: STOPPED
I20260926 07:35:33.958972 11401 leader_election.cc:302] T 04be706330924282961054c5835cec73 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 22f3902d01584936b20437606cb86dd1; no voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:33.959860 12212 raft_consensus.cc:2726] T 04be706330924282961054c5835cec73 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:33.915103 11462 raft_consensus.cc:3037] T 3e28ca159d8349748812825ac1947c51 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:33.930123 12256 tablet.cc:1722] T ae16af9edff9401589bcf54eeb3e8dfe P 22f3902d01584936b20437606cb86dd1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 07:35:33.880645 11113 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: default.Table_72783eb75b984ed394613e1333b94227 [id=65c112d56f2a44cc9d840caadd218412] alter complete (version 1)
I20260926 07:35:33.973536 12212 consensus_queue.cc:1035] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 07:35:33.975529 11401 leader_election.cc:341] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
W20260926 07:35:33.976660 11401 leader_election.cc:341] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:33.976958 11746 tablet_bootstrap.cc:654] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:33.976658 11114 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_c3edf5d221b54d5b86c24973e57cec68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:33.977712 11401 leader_election.cc:341] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
W20260926 07:35:33.978150 11114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c3edf5d221b54d5b86c24973e57cec68 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:33.983218 11871 ts_tablet_manager.cc:1939] T 9feaf1ed8ff34928b716fd0b4e41cdde P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
W20260926 07:35:33.989276 11401 leader_election.cc:341] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:33.990316 11401 leader_election.cc:302] T 3e28ca159d8349748812825ac1947c51 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 22f3902d01584936b20437606cb86dd1; no voters: 6070e9e3d1234a9d83e49431416afade, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:33.991264 11445 tablet_service.cc:1467] Processing CreateTablet for tablet 5f2ef7ebe6e144529cffd699180d5be8 (DEFAULT_TABLE table=default.Table_4d993e44ddc44452a2be1c319f81f9e8 [id=b359877938204095b5ed3cfbe94997a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:33.996515 11115 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: default.Table_d6cdabd37ad94b339606911c37764fba [id=55a6e4b7598043d8bf49512f2542c589] alter complete (version 4)
I20260926 07:35:33.968796 11660 heartbeater.cc:503] Master 127.10.171.62:37629 was elected leader, sending a full tablet report...
I20260926 07:35:33.998467 12212 raft_consensus.cc:2726] T 3e28ca159d8349748812825ac1947c51 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:34.022081 11439 tablet_service.cc:1467] Processing CreateTablet for tablet 0b900e55dcb34eef9f76de31b389e12d (DEFAULT_TABLE table=default.Table_81f434ad83d24e2b9abc65b59e28381e [id=965045547daf4502814c7296026c3187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.022606 11435 tablet_service.cc:1514] Processing DeleteTablet for tablet 05f071b4db7847a5adbd25a71cac3ea7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:30 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:34.023108 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet c81a59361a6e42b7962b3e027c4a087f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:28 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:34.023736 11436 tablet_service.cc:1514] Processing DeleteTablet for tablet 509f607b089c44bc9c3f80ed59a68c95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:34.023697 11435 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e1cea4523a64428aac03ca0bdf708c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:24 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:34.024400 11437 tablet_service.cc:1514] Processing DeleteTablet for tablet 55ad1d96e2314d1eb0a717a3e9ff31bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:34.025593 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet d6fc6178f1e54c3984a20873864233cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:26 UTC) from {username='slave'} at 127.0.0.1:40244
W20260926 07:35:34.027848 11117 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.3:53335 (request call id 30) took 2139 ms (client timeout 1000 ms). Trace:
I20260926 07:35:34.028143 11515 heartbeater.cc:347] Connected to a master server at 127.10.171.62:37629
I20260926 07:35:34.032428 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet a0bdf6a4656e4217b0a7013fddc4cca8 with delete_type TABLET_DATA_DELETED (replaced by tablet 089bac5feeaa4508bc863e446b172033) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:34.033351 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet a8b629a4aba045b697e33bbebecb05b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:34.033998 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet 4239f5d48512403bb711ece401f0756b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:28 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:34.035189 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet c36c759963cb4915afdca7d12109cc6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:27 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:34.035754 11437 tablet_service.cc:1514] Processing DeleteTablet for tablet 04180ace5d4548dfbcd226bf229d5c26 with delete_type TABLET_DATA_DELETED (replaced by tablet b862f325a9df49079066dc62fe1a61f1) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:34.037701 11437 tablet_service.cc:1514] Processing DeleteTablet for tablet 034d508f0a164f3ea8308ea2d28818c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 4088ccb81e844cc29e1735bb341e4a92) from {username='slave'} at 127.0.0.1:40244
W20260926 07:35:34.031273 11113 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.1:34359 (request call id 45) took 1365 ms (client timeout 1000 ms). Trace:
W20260926 07:35:34.035915 11119 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.2:33317 (request call id 48) took 1205 ms (client timeout 1000 ms). Trace:
W20260926 07:35:34.038085 11115 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.2:33317 (request call id 47) took 2219 ms (client timeout 1000 ms). Trace:
W20260926 07:35:34.038789 11117 rpcz_store.cc:269] 0926 07:35:31.888177 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:31.888416 (+ 239us) service_pool.cc:227] Handling call
0926 07:35:34.027783 (+2139367us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:32.772480 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:32.772849 (+ 369us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:32.772886 (+ 37us) write_op.cc:435] Acquired schema lock
0926 07:35:32.772900 (+ 14us) tablet.cc:660] Decoding operations
0926 07:35:32.773133 (+ 233us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:32.773174 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:32.773198 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0926 07:35:32.773326 (+ 128us) tablet.cc:699] Row locks acquired
0926 07:35:32.773341 (+ 15us) write_op.cc:260] PREPARE: finished
0926 07:35:32.773432 (+ 91us) write_op.cc:270] Start()
0926 07:35:32.773522 (+ 90us) write_op.cc:276] Timestamp: P: 1790408132773384 usec, L: 0
0926 07:35:32.773540 (+ 18us) op_driver.cc:331] REPLICATION: starting
0926 07:35:32.774741 (+ 1201us) log.cc:842] Serialized 530 byte log entry
0926 07:35:32.792930 (+ 18189us) op_driver.cc:444] REPLICATION: finished
0926 07:35:32.793802 (+ 872us) write_op.cc:301] APPLY: starting
0926 07:35:32.793869 (+ 67us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:32.793964 (+ 95us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:32.794083 (+ 119us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:32.794236 (+ 153us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:32.794366 (+ 130us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:32.794418 (+ 52us) write_op.cc:312] APPLY: finished
0926 07:35:32.794591 (+ 173us) log.cc:842] Serialized 23 byte log entry
0926 07:35:32.794737 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:32.794799 (+ 62us) write_op.cc:454] Released schema lock
0926 07:35:32.794832 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:32.795009 (+ 177us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:32.795036 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":11956,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":724,"apply.run_cpu_time_us":1380,"apply.run_wall_time_us":1380,"num_ops":1,"prepare.queue_time_us":99103,"prepare.run_cpu_time_us":1836,"prepare.run_wall_time_us":3365,"raft.queue_time_us":2798,"raft.run_cpu_time_us":709,"raft.run_wall_time_us":13759,"replication_time_us":19276,"spinlock_wait_cycles":1804032}]]}
W20260926 07:35:34.054893 11545 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.171.2:42379, user_credentials={real_user=slave}} blocked reactor thread for 56509.4us
I20260926 07:35:34.067262 11746 tablet_bootstrap.cc:492] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:34.067701 11746 ts_tablet_manager.cc:1397] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.112s user 0.014s sys 0.000s
I20260926 07:35:34.068765 11591 tablet_service.cc:1467] Processing CreateTablet for tablet 0b900e55dcb34eef9f76de31b389e12d (DEFAULT_TABLE table=default.Table_81f434ad83d24e2b9abc65b59e28381e [id=965045547daf4502814c7296026c3187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.077780 11295 tablet_service.cc:1514] Processing DeleteTablet for tablet 640d21586bad4070957cff61bd174fd2 with delete_type TABLET_DATA_DELETED (replaced by tablet 6d54a5c32da5444ca0a2939461f3a0f6) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:34.077924 11289 tablet_service.cc:1514] Processing DeleteTablet for tablet 05f071b4db7847a5adbd25a71cac3ea7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:30 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:34.082412 11437 tablet_service.cc:1467] Processing CreateTablet for tablet 7019af1230224e999e078b21ea5a30a5 (DEFAULT_TABLE table=default.Table_d0f2927c94ad48919b98e4f4199b42cd [id=31ef8b4e5e134c96877c593f57a38376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.084931 11442 tablet_service.cc:1467] Processing CreateTablet for tablet ed0224dadfae4f58895107d205277b30 (DEFAULT_TABLE table=default.Table_8b58d5bf0cf74a2f812cf5f7602515b8 [id=688fcf7e1732465c9ad4d96ee8bb8621]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.087448 11589 tablet_service.cc:1514] Processing DeleteTablet for tablet 4239f5d48512403bb711ece401f0756b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:28 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:34.088441 11295 tablet_service.cc:1514] Processing DeleteTablet for tablet 034d508f0a164f3ea8308ea2d28818c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 4088ccb81e844cc29e1735bb341e4a92) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:34.088523 11293 tablet_service.cc:1514] Processing DeleteTablet for tablet d6fc6178f1e54c3984a20873864233cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:26 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:34.077775 11286 tablet_service.cc:1514] Processing DeleteTablet for tablet 4088ccb81e844cc29e1735bb341e4a92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:33 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:34.089545 11289 tablet_service.cc:1514] Processing DeleteTablet for tablet c36c759963cb4915afdca7d12109cc6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:27 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:34.102942 11589 tablet_service.cc:1514] Processing DeleteTablet for tablet 04be706330924282961054c5835cec73 with delete_type TABLET_DATA_DELETED (replaced by tablet 21919d22b1d24546911d36384b01d9cc) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:34.106599 11588 tablet_service.cc:1467] Processing CreateTablet for tablet 5f2ef7ebe6e144529cffd699180d5be8 (DEFAULT_TABLE table=default.Table_4d993e44ddc44452a2be1c319f81f9e8 [id=b359877938204095b5ed3cfbe94997a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.101289 11581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f3986a19a5044159c89ae6ba72b247c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.114156 11587 tablet_service.cc:1514] Processing DeleteTablet for tablet c36c759963cb4915afdca7d12109cc6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:27 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:34.114311 11586 tablet_service.cc:1467] Processing CreateTablet for tablet ed0224dadfae4f58895107d205277b30 (DEFAULT_TABLE table=default.Table_8b58d5bf0cf74a2f812cf5f7602515b8 [id=688fcf7e1732465c9ad4d96ee8bb8621]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.115494 11586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed0224dadfae4f58895107d205277b30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.115339 11580 tablet_service.cc:1467] Processing CreateTablet for tablet 7019af1230224e999e078b21ea5a30a5 (DEFAULT_TABLE table=default.Table_d0f2927c94ad48919b98e4f4199b42cd [id=31ef8b4e5e134c96877c593f57a38376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.116428 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7019af1230224e999e078b21ea5a30a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.120908 11588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f2ef7ebe6e144529cffd699180d5be8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.070195 12133 consensus_queue.cc:1035] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260926 07:35:34.127648 11314 raft_consensus.cc:2454] T 4239f5d48512403bb711ece401f0756b P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
W20260926 07:35:34.134752 11545 consensus_peers.cc:487] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade -> Peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Couldn't send request to peer 22f3902d01584936b20437606cb86dd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a0bdf6a4656e4217b0a7013fddc4cca8. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:34.152688 12212 consensus_queue.cc:1035] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:34.156175 11591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b900e55dcb34eef9f76de31b389e12d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:34.040863 11115 rpcz_store.cc:269] 0926 07:35:31.819150 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:31.844453 (+ 25303us) service_pool.cc:227] Handling call
0926 07:35:31.870679 (+ 26226us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812374c3e 00007fb80feabfb4 00007fb80feabf42 00007fb80feabe71 00007fb80feabdec 00007fb80feab0a2 00007fb80e15594c 00007fb80e154d86
0926 07:35:34.038027 (+2167348us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:33.934677 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:33.935008 (+ 331us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:33.935029 (+ 21us) write_op.cc:435] Acquired schema lock
0926 07:35:33.935041 (+ 12us) tablet.cc:660] Decoding operations
0926 07:35:33.935298 (+ 257us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:33.935353 (+ 55us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:33.935374 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0926 07:35:33.935518 (+ 144us) tablet.cc:699] Row locks acquired
0926 07:35:33.935531 (+ 13us) write_op.cc:260] PREPARE: finished
0926 07:35:33.935627 (+ 96us) write_op.cc:270] Start()
0926 07:35:33.935703 (+ 76us) write_op.cc:276] Timestamp: P: 1790408133935562 usec, L: 0
0926 07:35:33.935719 (+ 16us) op_driver.cc:331] REPLICATION: starting
0926 07:35:33.979458 (+ 43739us) log.cc:842] Serialized 356 byte log entry
0926 07:35:33.981542 (+ 2084us) spinlock_profiling.cc:243] Waited 43.2 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032b471 00007fb8103a2198 00007fb810803ade 00007fb8108032b8 00007fb810807ba1 00007fb810807b59 00007fb810807ae9 00007fb810807ab1 00007fb810806dad 0000000000580784 00000000005805b9
0926 07:35:33.989305 (+ 7763us) op_driver.cc:444] REPLICATION: finished
0926 07:35:33.995222 (+ 5917us) write_op.cc:301] APPLY: starting
0926 07:35:33.995284 (+ 62us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:33.995407 (+ 123us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:33.995434 (+ 27us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:33.995605 (+ 171us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:33.995736 (+ 131us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:33.995780 (+ 44us) write_op.cc:312] APPLY: finished
0926 07:35:33.995937 (+ 157us) log.cc:842] Serialized 23 byte log entry
0926 07:35:33.996055 (+ 118us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:33.996129 (+ 74us) write_op.cc:454] Released schema lock
0926 07:35:33.996162 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:33.996327 (+ 165us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:33.996351 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2310,"reported_tablets":17,"spinlock_wait_cycles":54060416,"child_traces":[["op",{"apply.queue_time_us":5325,"apply.run_cpu_time_us":1271,"apply.run_wall_time_us":1272,"num_ops":1,"prepare.queue_time_us":240813,"raft.queue_time_us":2148,"raft.run_cpu_time_us":558,"raft.run_wall_time_us":1299,"replication_time_us":53513,"spinlock_wait_cycles":99288960}]]}
W20260926 07:35:34.040753 11119 rpcz_store.cc:269] 0926 07:35:32.830580 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:32.830722 (+ 142us) service_pool.cc:227] Handling call
0926 07:35:33.054033 (+223311us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812374c3e 00007fb80feabfb4 00007fb80feabf42 00007fb80feabe71 00007fb80feabdec 00007fb80feab0a2 00007fb80e15594c 00007fb80e154d86
0926 07:35:34.035848 (+981815us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:33.600945 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:33.601276 (+ 331us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:33.601324 (+ 48us) write_op.cc:435] Acquired schema lock
0926 07:35:33.601338 (+ 14us) tablet.cc:660] Decoding operations
0926 07:35:33.601578 (+ 240us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:33.601615 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:33.601636 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0926 07:35:33.601766 (+ 130us) tablet.cc:699] Row locks acquired
0926 07:35:33.601779 (+ 13us) write_op.cc:260] PREPARE: finished
0926 07:35:33.601870 (+ 91us) write_op.cc:270] Start()
0926 07:35:33.601942 (+ 72us) write_op.cc:276] Timestamp: P: 1790408133601797 usec, L: 0
0926 07:35:33.601957 (+ 15us) op_driver.cc:331] REPLICATION: starting
0926 07:35:33.602403 (+ 446us) log.cc:842] Serialized 530 byte log entry
0926 07:35:33.618736 (+ 16333us) op_driver.cc:444] REPLICATION: finished
0926 07:35:33.619048 (+ 312us) write_op.cc:301] APPLY: starting
0926 07:35:33.619099 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:33.619183 (+ 84us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:33.619281 (+ 98us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:33.619425 (+ 144us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:33.619531 (+ 106us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:33.619570 (+ 39us) write_op.cc:312] APPLY: finished
0926 07:35:33.619710 (+ 140us) log.cc:842] Serialized 23 byte log entry
0926 07:35:33.619843 (+ 133us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:33.619897 (+ 54us) write_op.cc:454] Released schema lock
0926 07:35:33.619928 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:33.620070 (+ 142us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:33.620092 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":16,"spinlock_wait_cycles":505683328,"child_traces":[["op",{"apply.queue_time_us":182,"apply.run_cpu_time_us":1200,"apply.run_wall_time_us":6118,"num_ops":1,"prepare.queue_time_us":4558,"prepare.run_cpu_time_us":1822,"prepare.run_wall_time_us":37807,"raft.queue_time_us":1592,"raft.run_cpu_time_us":511,"raft.run_wall_time_us":1200,"replication_time_us":16718}]]}
W20260926 07:35:34.189280 11118 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5f2ef7ebe6e144529cffd699180d5be8 consensus state has no leader
I20260926 07:35:34.190806 11118 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_405e87138f64473890e7ba5b74d483eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:34.192607 11118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_405e87138f64473890e7ba5b74d483eb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:34.199966 11289 tablet_service.cc:1467] Processing CreateTablet for tablet 5f2ef7ebe6e144529cffd699180d5be8 (DEFAULT_TABLE table=default.Table_4d993e44ddc44452a2be1c319f81f9e8 [id=b359877938204095b5ed3cfbe94997a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.201041 11289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f2ef7ebe6e144529cffd699180d5be8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.209113 11756 tablet_bootstrap.cc:492] T 90f62593b6f844fc8a41b9be240101fd P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:34.209606 11756 ts_tablet_manager.cc:1397] T 90f62593b6f844fc8a41b9be240101fd P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.856s user 0.007s sys 0.011s
I20260926 07:35:34.211720 11286 tablet_service.cc:1467] Processing CreateTablet for tablet 0b900e55dcb34eef9f76de31b389e12d (DEFAULT_TABLE table=default.Table_81f434ad83d24e2b9abc65b59e28381e [id=965045547daf4502814c7296026c3187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.212931 11286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b900e55dcb34eef9f76de31b389e12d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.219229 11746 raft_consensus.cc:348] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:34.219946 11746 raft_consensus.cc:374] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:34.220305 11746 raft_consensus.cc:729] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:34.221138 11746 consensus_queue.cc:260] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
W20260926 07:35:34.234189 11105 catalog_manager.cc:4808] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 05f071b4db7847a5adbd25a71cac3ea7 because the tablet was not found. No further retry: Not found: Tablet not found: 05f071b4db7847a5adbd25a71cac3ea7
I20260926 07:35:34.255811 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 122) took 4573 ms. Trace:
W20260926 07:35:34.040622 11113 rpcz_store.cc:269] 0926 07:35:32.665368 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:32.669653 (+ 4285us) service_pool.cc:227] Handling call
0926 07:35:32.745472 (+ 75819us) spinlock_profiling.cc:243] Waited 73.5 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812374c3e 00007fb80feabfb4 00007fb80feabf42 00007fb80feabe71 00007fb80feabdec 00007fb80feab0a2 00007fb80e15594c 00007fb80e154d86
0926 07:35:34.031197 (+1285725us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:33.563077 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:33.563390 (+ 313us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:33.563410 (+ 20us) write_op.cc:435] Acquired schema lock
0926 07:35:33.563421 (+ 11us) tablet.cc:660] Decoding operations
0926 07:35:33.563636 (+ 215us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:33.563671 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:33.563690 (+ 19us) tablet.cc:683] Acquiring locks for 1 operations
0926 07:35:33.563806 (+ 116us) tablet.cc:699] Row locks acquired
0926 07:35:33.563817 (+ 11us) write_op.cc:260] PREPARE: finished
0926 07:35:33.563907 (+ 90us) write_op.cc:270] Start()
0926 07:35:33.563980 (+ 73us) write_op.cc:276] Timestamp: P: 1790408133563836 usec, L: 0
0926 07:35:33.563994 (+ 14us) op_driver.cc:331] REPLICATION: starting
0926 07:35:33.564478 (+ 484us) log.cc:842] Serialized 530 byte log entry
0926 07:35:33.570956 (+ 6478us) op_driver.cc:444] REPLICATION: finished
0926 07:35:33.575378 (+ 4422us) write_op.cc:301] APPLY: starting
0926 07:35:33.575437 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:33.575520 (+ 83us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:33.575599 (+ 79us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:33.575766 (+ 167us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:33.575888 (+ 122us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:33.575930 (+ 42us) write_op.cc:312] APPLY: finished
0926 07:35:33.576080 (+ 150us) log.cc:842] Serialized 23 byte log entry
0926 07:35:33.576236 (+ 156us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:33.576290 (+ 54us) write_op.cc:454] Released schema lock
0926 07:35:33.576318 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:33.576469 (+ 151us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:33.576501 (+ 32us) write_op.cc:454] Released schema lock
Related trace 'op':
0926 07:35:33.715693 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:33.716054 (+ 361us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:33.716076 (+ 22us) write_op.cc:435] Acquired schema lock
0926 07:35:33.716090 (+ 14us) tablet.cc:660] Decoding operations
0926 07:35:33.716382 (+ 292us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:33.716420 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:33.716444 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0926 07:35:33.815951 (+ 99507us) tablet.cc:699] Row locks acquired
0926 07:35:33.816036 (+ 85us) write_op.cc:260] PREPARE: finished
0926 07:35:33.816800 (+ 764us) write_op.cc:270] Start()
0926 07:35:33.817260 (+ 460us) write_op.cc:276] Timestamp: P: 1790408133816486 usec, L: 0
0926 07:35:33.817340 (+ 80us) op_driver.cc:331] REPLICATION: starting
0926 07:35:33.861068 (+ 43728us) log.cc:842] Serialized 356 byte log entry
0926 07:35:33.874285 (+ 13217us) op_driver.cc:444] REPLICATION: finished
0926 07:35:33.879136 (+ 4851us) write_op.cc:301] APPLY: starting
0926 07:35:33.879188 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:33.879267 (+ 79us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:33.879378 (+ 111us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:33.879528 (+ 150us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:33.879649 (+ 121us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:33.879690 (+ 41us) write_op.cc:312] APPLY: finished
0926 07:35:33.879840 (+ 150us) log.cc:842] Serialized 23 byte log entry
0926 07:35:33.879969 (+ 129us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:33.880027 (+ 58us) write_op.cc:454] Released schema lock
0926 07:35:33.880052 (+ 25us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:33.880224 (+ 172us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:33.880245 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":117815,"reported_tablets":14,"spinlock_wait_cycles":169071872,"child_traces":[["op",{"apply.queue_time_us":4281,"apply.run_cpu_time_us":1267,"apply.run_wall_time_us":1267,"num_ops":1,"prepare.queue_time_us":82832,"prepare.run_cpu_time_us":1679,"prepare.run_wall_time_us":37745,"raft.queue_time_us":77,"raft.run_cpu_time_us":603,"raft.run_wall_time_us":1448,"replication_time_us":6904}],["op",{"apply.queue_time_us":4711,"apply.run_cpu_time_us":1244,"apply.run_wall_time_us":1242,"num_ops":1,"prepare.queue_time_us":33203,"prepare.run_cpu_time_us":3292,"prepare.run_wall_time_us":218889,"raft.queue_time_us":93,"raft.run_cpu_time_us":538,"raft.run_wall_time_us":537,"replication_time_us":56812}]]}
I20260926 07:35:34.256256 11871 rpcz_store.cc:276] 0926 07:35:29.682392 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.682544 (+ 152us) service_pool.cc:227] Handling call
0926 07:35:33.387483 (+3704939us) ts_tablet_manager.cc:1145] Deleting tablet 9feaf1ed8ff34928b716fd0b4e41cdde
0926 07:35:33.387532 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:33.622934 (+235402us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:33.929735 (+306801us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:34.255486 (+325751us) spinlock_profiling.cc:243] Waited 57.1 ms on lock 0x7b5800001898. stack: 00007f9f9373c033 00007f9f9373bf30 00007f9f932bd534 00000000004db2b4 00000000004db249 00007f9f935b2f11 00007f9f93736851 00007f9f93736e82 00007f9f9c5f68b9 00007f9f9b62a5d8 00007f9f9b6eee42 00007f9f9b71ee10 00007f9f9b705e0d 00007f9f9b705db9 00007f9f9b705d49 00007f9f9b705d11
0926 07:35:34.255758 (+ 272us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":131331968,"tablet-delete.queue_time_us":3704305}
I20260926 07:35:34.257454 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 9feaf1ed8ff34928b716fd0b4e41cdde (table default.Table_783e2f568d774b5eb908860af5fa4166 [id=d5913e299402470898908b32cdba262a]) successfully deleted
I20260926 07:35:34.258628 11871 tablet_replica.cc:330] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:34.280522 12223 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6e2fc900b374ebab759831b571a208d. Latest consensus state: current_term: 3 leader_uuid: "b6e2fc900b374ebab759831b571a208d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } } }
I20260926 07:35:34.281500 12223 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:35:34.285966 12220 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b6e2fc900b374ebab759831b571a208d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } } }
I20260926 07:35:34.286865 12220 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:35:34.287623 12258 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b6e2fc900b374ebab759831b571a208d)
I20260926 07:35:34.288396 12258 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:34.292378 11586 tablet_service.cc:1467] Processing CreateTablet for tablet 4088ccb81e844cc29e1735bb341e4a92 (DEFAULT_TABLE table=default.Table_53678f9c4c01403eb2d846d3242be0aa [id=db19d49c9c29488e89f7280e91a11713]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.293745 11586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4088ccb81e844cc29e1735bb341e4a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.293220 12271 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "b6e2fc900b374ebab759831b571a208d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } } }
I20260926 07:35:34.294155 12271 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 07:35:34.295929 11106 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 6c42b518ef04429aa6b4e8f5deee05c8 (table default.Table_3de17a801f1c4fa99878e07e9df543f8 [id=0ebbd3b8ecd242c2a21e0630227a01c7], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 6c42b518ef04429aa6b4e8f5deee05c8 consensus state has no leader
I20260926 07:35:34.318497 11754 tablet_bootstrap.cc:492] T 17871fbd15064daba6e80a34a1b221d3 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:34.318984 11754 ts_tablet_manager.cc:1397] T 17871fbd15064daba6e80a34a1b221d3 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.847s user 0.015s sys 0.002s
I20260926 07:35:34.325105 12258 leader_election.cc:288] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0773380643194748af6701300513e7a9 (127.10.171.61:38823), b6e2fc900b374ebab759831b571a208d (127.10.171.60:40073)
I20260926 07:35:34.340616 11871 raft_consensus.cc:2229] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:34.341122 11871 raft_consensus.cc:2258] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:34.342696 11871 ts_tablet_manager.cc:1905] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:34.345592 11115 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_ed9c48cda45b4d4fbf739fbea19e7943"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:34.328678 11754 raft_consensus.cc:348] T 17871fbd15064daba6e80a34a1b221d3 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:34.374049 11754 raft_consensus.cc:374] T 17871fbd15064daba6e80a34a1b221d3 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:34.375659 12071 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38292 (request call id 0) took 2664 ms. Trace:
I20260926 07:35:34.231060 11445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f2ef7ebe6e144529cffd699180d5be8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.232059 12259 tablet_replica.cc:330] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:34.237280 11435 tablet_service.cc:1467] Processing CreateTablet for tablet 4088ccb81e844cc29e1735bb341e4a92 (DEFAULT_TABLE table=default.Table_53678f9c4c01403eb2d846d3242be0aa [id=db19d49c9c29488e89f7280e91a11713]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.239167 11436 tablet_service.cc:1467] Processing CreateTablet for tablet 2dd5175f0ca941f8b3856ea415e2ee68 (DEFAULT_TABLE table=default.Table_fd842e14c4984402b2b1b77baa1f9360 [id=cc4663f9d1734d9cb4cc4e1f061310a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.243522 12212 raft_consensus.cc:482] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:34.382826 12212 raft_consensus.cc:504] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:34.382115 11435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4088ccb81e844cc29e1735bb341e4a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.387883 12259 raft_consensus.cc:2229] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:34.388326 12259 raft_consensus.cc:2258] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:34.388541 11436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dd5175f0ca941f8b3856ea415e2ee68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.390120 12259 ts_tablet_manager.cc:1905] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:34.397269 12212 leader_election.cc:288] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:34.398236 11442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed0224dadfae4f58895107d205277b30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.403956 11437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7019af1230224e999e078b21ea5a30a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.405064 11439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b900e55dcb34eef9f76de31b389e12d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.414821 11754 raft_consensus.cc:729] T 17871fbd15064daba6e80a34a1b221d3 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:34.433576 11754 consensus_queue.cc:260] T 17871fbd15064daba6e80a34a1b221d3 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:34.435559 11754 ts_tablet_manager.cc:1428] T 17871fbd15064daba6e80a34a1b221d3 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.116s user 0.006s sys 0.000s
I20260926 07:35:34.336349 12237 raft_consensus.cc:482] T c81a59361a6e42b7962b3e027c4a087f P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:34.450739 11294 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e28ca159d8349748812825ac1947c51 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dd5175f0ca941f8b3856ea415e2ee68) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:34.452324 11293 tablet_service.cc:1467] Processing CreateTablet for tablet 7019af1230224e999e078b21ea5a30a5 (DEFAULT_TABLE table=default.Table_d0f2927c94ad48919b98e4f4199b42cd [id=31ef8b4e5e134c96877c593f57a38376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.448779 12237 raft_consensus.cc:504] T c81a59361a6e42b7962b3e027c4a087f P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:34.463620 12237 leader_election.cc:288] T c81a59361a6e42b7962b3e027c4a087f P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:34.465607 11293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7019af1230224e999e078b21ea5a30a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.472900 11754 tablet_bootstrap.cc:492] T ed9f7faf55684e3e8512a033aa1443fe P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:34.482074 11756 raft_consensus.cc:348] T 90f62593b6f844fc8a41b9be240101fd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:34.482801 11756 raft_consensus.cc:374] T 90f62593b6f844fc8a41b9be240101fd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:34.483070 11756 raft_consensus.cc:729] T 90f62593b6f844fc8a41b9be240101fd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:34.508694 11756 consensus_queue.cc:260] T 90f62593b6f844fc8a41b9be240101fd P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:34.514616 11871 ts_tablet_manager.cc:1918] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:34.518282 11871 log.cc:1196] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/afa901cdc6a84220bf36ee669a955b72
I20260926 07:35:34.518954 11871 ts_tablet_manager.cc:1939] T afa901cdc6a84220bf36ee669a955b72 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:34.519701 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 123) took 4825 ms. Trace:
I20260926 07:35:34.519964 11871 rpcz_store.cc:276] 0926 07:35:29.694478 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.694593 (+ 115us) service_pool.cc:227] Handling call
0926 07:35:34.258484 (+4563891us) ts_tablet_manager.cc:1145] Deleting tablet afa901cdc6a84220bf36ee669a955b72
0926 07:35:34.258524 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:34.461752 (+203228us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:34.514376 (+ 52624us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:34.519652 (+ 5276us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":58855,"tablet-delete.queue_time_us":4558199}
W20260926 07:35:34.528671 11115 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ed9c48cda45b4d4fbf739fbea19e7943 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:34.534288 11583 tablet_service.cc:1467] Processing CreateTablet for tablet 2dd5175f0ca941f8b3856ea415e2ee68 (DEFAULT_TABLE table=default.Table_fd842e14c4984402b2b1b77baa1f9360 [id=cc4663f9d1734d9cb4cc4e1f061310a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.535369 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dd5175f0ca941f8b3856ea415e2ee68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.493175 11296 tablet_service.cc:1467] Processing CreateTablet for tablet 3f3986a19a5044159c89ae6ba72b247c (DEFAULT_TABLE table=default.Table_53f51dc492ee41569d26cfe526418f7b [id=c4422fedd4854d4d94a16d9fb3311e53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.494674 11294 tablet_service.cc:1467] Processing CreateTablet for tablet ed0224dadfae4f58895107d205277b30 (DEFAULT_TABLE table=default.Table_8b58d5bf0cf74a2f812cf5f7602515b8 [id=688fcf7e1732465c9ad4d96ee8bb8621]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.547940 11746 ts_tablet_manager.cc:1428] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.480s user 0.006s sys 0.000s
I20260926 07:35:34.547519 11581 tablet_service.cc:1467] Processing CreateTablet for tablet e219b112c1a24bca84befe262adfee19 (DEFAULT_TABLE table=default.Table_d605de609d19450cbe6cf9af7b9774fd [id=d14bc02fec09496db9a0d56d739b5208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.555810 11580 tablet_service.cc:1467] Processing CreateTablet for tablet 4081354696234f33b51bb3c0d141f7e6 (DEFAULT_TABLE table=default.Table_11baf2ea5fdc431fae1ae6d0dd3cdb2b [id=1c14eed77a104f2ba5e306f94bffd72a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.552369 11871 tablet_replica.cc:330] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:34.565867 11581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e219b112c1a24bca84befe262adfee19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.567870 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4081354696234f33b51bb3c0d141f7e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.545118 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet afa901cdc6a84220bf36ee669a955b72 (table default.Table_b5e830fcbaa44c209d50a84038d3b68f [id=be6bd29e04f342cbb7993b18e13d9332]) successfully deleted
I20260926 07:35:34.578259 11588 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cd482fb159d418291cb02bc9138b076 with delete_type TABLET_DATA_DELETED (replaced by tablet 393da59b3671473ca9714d3406dd1388) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:34.579049 11591 tablet_service.cc:1514] Processing DeleteTablet for tablet 9feaf1ed8ff34928b716fd0b4e41cdde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:29 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:34.587849 12202 raft_consensus.cc:482] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9bfebb34fa14cd2ad4843fffbe5c54e)
I20260926 07:35:34.588335 12202 raft_consensus.cc:504] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } }
I20260926 07:35:34.590116 12202 leader_election.cc:288] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:34.593559 11294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed0224dadfae4f58895107d205277b30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.596712 11871 raft_consensus.cc:2229] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:34.596845 12284 tablet.cc:1722] T ae16af9edff9401589bcf54eeb3e8dfe P 6070e9e3d1234a9d83e49431416afade: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 07:35:34.496387 11295 tablet_service.cc:1467] Processing CreateTablet for tablet 4088ccb81e844cc29e1735bb341e4a92 (DEFAULT_TABLE table=default.Table_53678f9c4c01403eb2d846d3242be0aa [id=db19d49c9c29488e89f7280e91a11713]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.597424 11871 raft_consensus.cc:2258] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:34.598233 11295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4088ccb81e844cc29e1735bb341e4a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.599172 11871 ts_tablet_manager.cc:1905] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:34.498080 11287 tablet_service.cc:1467] Processing CreateTablet for tablet 2dd5175f0ca941f8b3856ea415e2ee68 (DEFAULT_TABLE table=default.Table_fd842e14c4984402b2b1b77baa1f9360 [id=cc4663f9d1734d9cb4cc4e1f061310a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.600373 11754 tablet_bootstrap.cc:654] T ed9f7faf55684e3e8512a033aa1443fe P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:34.600775 11287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dd5175f0ca941f8b3856ea415e2ee68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.499691 11291 tablet_service.cc:1467] Processing CreateTablet for tablet 4081354696234f33b51bb3c0d141f7e6 (DEFAULT_TABLE table=default.Table_11baf2ea5fdc431fae1ae6d0dd3cdb2b [id=1c14eed77a104f2ba5e306f94bffd72a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.604696 11291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4081354696234f33b51bb3c0d141f7e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.501312 11284 tablet_service.cc:1467] Processing CreateTablet for tablet e219b112c1a24bca84befe262adfee19 (DEFAULT_TABLE table=default.Table_d605de609d19450cbe6cf9af7b9774fd [id=d14bc02fec09496db9a0d56d739b5208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.492280 11285 tablet_service.cc:1467] Processing CreateTablet for tablet 4d0ac73eb239499083eead51e0a7e229 (DEFAULT_TABLE table=default.Table_e0f192a8a56f4343afa3cb8b77b6148d [id=f47c787764724aec9b36787e78b256bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.627060 11120 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_ba602ce39d464e51bd0faa02b98f4363" } modify_external_catalogs: true
I20260926 07:35:34.631887 11113 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_47fd16cdaf2a4eedbfb8a02517543f80" } new_table_name: "default.Table_2b10a1845a4044a8a8ecce2c10436ae6" modify_external_catalogs: true
I20260926 07:35:34.643324 11754 tablet_bootstrap.cc:492] T ed9f7faf55684e3e8512a033aa1443fe P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:34.643784 11754 ts_tablet_manager.cc:1397] T ed9f7faf55684e3e8512a033aa1443fe P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.171s user 0.005s sys 0.007s
I20260926 07:35:34.652205 12133 raft_consensus.cc:482] T 17871fbd15064daba6e80a34a1b221d3 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:34.652900 12133 raft_consensus.cc:504] T 17871fbd15064daba6e80a34a1b221d3 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:34.657274 12133 leader_election.cc:288] T 17871fbd15064daba6e80a34a1b221d3 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:34.670639 11754 raft_consensus.cc:348] T ed9f7faf55684e3e8512a033aa1443fe P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:34.672675 11754 raft_consensus.cc:374] T ed9f7faf55684e3e8512a033aa1443fe P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:34.672936 11754 raft_consensus.cc:729] T ed9f7faf55684e3e8512a033aa1443fe P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:34.673477 11754 consensus_queue.cc:260] T ed9f7faf55684e3e8512a033aa1443fe P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:34.696512 12285 catalog_manager.cc:1259] Loaded cluster ID: d1a1bab5ae3b4961a68dafcdbb937fd6
I20260926 07:35:34.726470 11746 tablet_bootstrap.cc:492] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:34.732817 11754 ts_tablet_manager.cc:1428] T ed9f7faf55684e3e8512a033aa1443fe P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.089s user 0.005s sys 0.003s
I20260926 07:35:34.734047 11754 tablet_bootstrap.cc:492] T cef2e44be4084fc78e61cf822659b4ea P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:34.753638 12285 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249: loading cluster ID for follower catalog manager: success
W20260926 07:35:34.544041 11111 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.3:53335 (request call id 31) took 1375 ms (client timeout 1000 ms). Trace:
W20260926 07:35:34.761802 11111 rpcz_store.cc:269] 0926 07:35:33.168352 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:33.219592 (+ 51240us) service_pool.cc:227] Handling call
0926 07:35:33.229800 (+ 10208us) spinlock_profiling.cc:243] Waited 5.03 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812374c3e 00007fb80feabfb4 00007fb80feabf42 00007fb80feabe71 00007fb80feabdec 00007fb80feab0a2 00007fb80e15594c 00007fb80e154d86
0926 07:35:34.543957 (+1314157us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":10557,"reported_tablets":26,"spinlock_wait_cycles":11578240}
I20260926 07:35:34.765652 12038 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "22f3902d01584936b20437606cb86dd1" instance_seqno: 1790408117643137) as {username='slave'} at 127.10.171.2:54119; Asking this server to re-register.
I20260926 07:35:34.774082 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f581510fe624127a4d27d302f5be614" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
W20260926 07:35:34.790966 11462 thread.cc:636] tablet-copy [worker] (thread pool) Time spent creating pthread: real 0.666s user 0.038s sys 0.069s
W20260926 07:35:34.791292 11462 thread.cc:602] tablet-copy [worker] (thread pool) Time spent starting thread: real 0.666s user 0.038s sys 0.069s
W20260926 07:35:34.793542 11402 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.171.3:45127, user_credentials={real_user=slave}} blocked reactor thread for 418237us
I20260926 07:35:34.786953 11051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6ef5b900d0f4e73a358f415efa97249" candidate_term: 4 candidate_status { last_received { term: 3 index: 205 } } ignore_live_leader: false dest_uuid: "0773380643194748af6701300513e7a9" is_pre_election: true
I20260926 07:35:34.802368 11130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6ef5b900d0f4e73a358f415efa97249" candidate_term: 4 candidate_status { last_received { term: 3 index: 205 } } ignore_live_leader: false dest_uuid: "b6e2fc900b374ebab759831b571a208d" is_pre_election: true
I20260926 07:35:34.804142 11120 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_ba602ce39d464e51bd0faa02b98f4363" } modify_external_catalogs: true
I20260926 07:35:34.804796 11112 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet f4aa8bef4bc64953ae2c573238838469
I20260926 07:35:34.805584 11116 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet c092aff8f2a041ccb09bc4b48e0f359c
I20260926 07:35:34.806924 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet c092aff8f2a041ccb09bc4b48e0f359c with delete_type TABLET_DATA_DELETED (replaced by tablet 191f35545a13402db05b71dafa589845) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:34.809465 11284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e219b112c1a24bca84befe262adfee19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:34.811606 11609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f581510fe624127a4d27d302f5be614" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:34.812206 11609 raft_consensus.cc:2454] T 3f581510fe624127a4d27d302f5be614 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:34.815548 11756 ts_tablet_manager.cc:1428] T 90f62593b6f844fc8a41b9be240101fd P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.606s user 0.004s sys 0.001s
I20260926 07:35:34.817219 12237 raft_consensus.cc:482] T 90f62593b6f844fc8a41b9be240101fd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:34.817641 12237 raft_consensus.cc:504] T 90f62593b6f844fc8a41b9be240101fd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:34.832226 12237 leader_election.cc:288] T 90f62593b6f844fc8a41b9be240101fd P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:34.836740 12133 consensus_queue.cc:1035] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:34.840711 11116 catalog_manager.cc:6565] ReplaceTablet: tablet c092aff8f2a041ccb09bc4b48e0f359c deleted and replaced by tablet 191f35545a13402db05b71dafa589845
I20260926 07:35:34.841131 11116 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:44836 (request call id 254) took 1181 ms. Trace:
I20260926 07:35:34.739969 11754 tablet_bootstrap.cc:654] T cef2e44be4084fc78e61cf822659b4ea P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:34.815832 11756 ts_tablet_manager.cc:1460] T 90f62593b6f844fc8a41b9be240101fd P e9bfebb34fa14cd2ad4843fffbe5c54e: Tablet startup took 605ms
W20260926 07:35:34.842114 11290 consensus_meta.cc:287] T dca3e232caf14ee88adf61d440113262 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent flushing consensus metadata: real 0.511s user 0.005s sys 0.000s
W20260926 07:35:34.845817 11288 consensus_meta.cc:287] T 246c13d624df4959b7b9e6e811da6818 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent flushing consensus metadata: real 0.616s user 0.005s sys 0.000s
W20260926 07:35:34.862231 11292 consensus_meta.cc:287] T 6acc4f70e461477a97d53136e14e4961 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent flushing consensus metadata: real 0.729s user 0.001s sys 0.006s
I20260926 07:35:34.876881 11112 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:44836 (request call id 256) took 1067 ms. Trace:
I20260926 07:35:34.884337 11981 leader_election.cc:302] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6ef5b900d0f4e73a358f415efa97249; no voters: 0773380643194748af6701300513e7a9, b6e2fc900b374ebab759831b571a208d
I20260926 07:35:34.872788 12253 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:52070 (request call id 148) took 2087 ms. Trace:
I20260926 07:35:34.904935 11871 ts_tablet_manager.cc:1918] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:34.907614 11288 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:52070 (request call id 152) took 1186 ms. Trace:
I20260926 07:35:34.908545 11871 log.cc:1196] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/1e1cea4523a64428aac03ca0bdf708c7
I20260926 07:35:34.908059 11288 rpcz_store.cc:276] 0926 07:35:33.705595 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:33.706811 (+ 1216us) service_pool.cc:227] Handling call
0926 07:35:33.762562 (+ 55751us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0926 07:35:33.762656 (+ 94us) ts_tablet_manager.cc:676] Creating new metadata...
0926 07:35:34.229006 (+466350us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:34.870725 (+641719us) tablet_replica.cc:205] Creating consensus instance
0926 07:35:34.891963 (+ 21238us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260926 07:35:34.908967 11292 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:52070 (request call id 154) took 1189 ms. Trace:
I20260926 07:35:34.909440 11292 rpcz_store.cc:276] 0926 07:35:33.706240 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:33.707017 (+ 777us) service_pool.cc:227] Handling call
0926 07:35:33.762989 (+ 55972us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0926 07:35:33.763063 (+ 74us) ts_tablet_manager.cc:676] Creating new metadata...
0926 07:35:34.131978 (+368915us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:34.872375 (+740397us) tablet_replica.cc:205] Creating consensus instance
0926 07:35:34.895860 (+ 23485us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260926 07:35:34.911029 11871 ts_tablet_manager.cc:1939] T 1e1cea4523a64428aac03ca0bdf708c7 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:34.924789 11114 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:44836 (request call id 255) took 1178 ms. Trace:
I20260926 07:35:34.925947 11754 tablet_bootstrap.cc:492] T cef2e44be4084fc78e61cf822659b4ea P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:34.926378 11754 ts_tablet_manager.cc:1397] T cef2e44be4084fc78e61cf822659b4ea P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.193s user 0.010s sys 0.003s
I20260926 07:35:34.925089 11114 rpcz_store.cc:276] 0926 07:35:33.746221 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:33.840610 (+ 94389us) service_pool.cc:227] Handling call
0926 07:35:33.976497 (+135887us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812376156 00007fb80feb3034 00007fb80feb2fc2 00007fb80feb2ef1 00007fb80feb2e6c 00007fb80feb2122 00007fb80e15594c 00007fb80e154d86
0926 07:35:33.982581 (+ 6084us) catalog_manager.cc:2075] Acquired catalog manager lock
0926 07:35:34.130745 (+148164us) catalog_manager.cc:2121] Created new table and tablet info
0926 07:35:34.924249 (+793504us) catalog_manager.cc:2181] Wrote table and tablets to system table
0926 07:35:34.924515 (+ 266us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0926 07:35:34.924739 (+ 224us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:34.802172 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:34.803875 (+ 1703us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:34.803899 (+ 24us) write_op.cc:435] Acquired schema lock
0926 07:35:34.803910 (+ 11us) tablet.cc:660] Decoding operations
0926 07:35:34.833289 (+ 29379us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:34.833334 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:34.833362 (+ 28us) tablet.cc:683] Acquiring locks for 2 operations
0926 07:35:34.833624 (+ 262us) tablet.cc:699] Row locks acquired
0926 07:35:34.833644 (+ 20us) write_op.cc:260] PREPARE: finished
0926 07:35:34.833789 (+ 145us) write_op.cc:270] Start()
0926 07:35:34.833896 (+ 107us) write_op.cc:276] Timestamp: P: 1790408134833714 usec, L: 0
0926 07:35:34.833918 (+ 22us) op_driver.cc:331] REPLICATION: starting
0926 07:35:34.834524 (+ 606us) log.cc:842] Serialized 433 byte log entry
0926 07:35:34.920515 (+ 85991us) op_driver.cc:444] REPLICATION: finished
0926 07:35:34.921963 (+ 1448us) write_op.cc:301] APPLY: starting
0926 07:35:34.922022 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:34.922113 (+ 91us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:34.922192 (+ 79us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:34.922535 (+ 343us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:34.922718 (+ 183us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 07:35:34.922774 (+ 56us) write_op.cc:312] APPLY: finished
0926 07:35:34.922937 (+ 163us) log.cc:842] Serialized 29 byte log entry
0926 07:35:34.923066 (+ 129us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:34.923122 (+ 56us) write_op.cc:454] Released schema lock
0926 07:35:34.923160 (+ 38us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:34.923312 (+ 152us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:34.923335 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":286819072,"child_traces":[["op",{"apply.queue_time_us":1253,"apply.run_cpu_time_us":1527,"apply.run_wall_time_us":1527,"num_ops":2,"prepare.queue_time_us":666304,"prepare.run_cpu_time_us":2374,"prepare.run_wall_time_us":58078,"raft.queue_time_us":99,"raft.run_cpu_time_us":1053,"raft.run_wall_time_us":1052,"replication_time_us":86512}]]}
I20260926 07:35:34.931507 11118 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:44836 (request call id 258) took 1011 ms. Trace:
I20260926 07:35:34.933621 12293 raft_consensus.cc:482] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:34.931843 11118 rpcz_store.cc:276] 0926 07:35:33.920044 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.190052 (+270008us) service_pool.cc:227] Handling call
0926 07:35:34.193067 (+ 3015us) catalog_manager.cc:2075] Acquired catalog manager lock
0926 07:35:34.194668 (+ 1601us) catalog_manager.cc:2121] Created new table and tablet info
0926 07:35:34.930942 (+736274us) catalog_manager.cc:2181] Wrote table and tablets to system table
0926 07:35:34.931201 (+ 259us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0926 07:35:34.931447 (+ 246us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:34.856720 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:34.857171 (+ 451us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:34.857202 (+ 31us) write_op.cc:435] Acquired schema lock
0926 07:35:34.857219 (+ 17us) tablet.cc:660] Decoding operations
0926 07:35:34.857772 (+ 553us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:34.857815 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:34.857852 (+ 37us) tablet.cc:683] Acquiring locks for 2 operations
0926 07:35:34.858063 (+ 211us) tablet.cc:699] Row locks acquired
0926 07:35:34.858076 (+ 13us) write_op.cc:260] PREPARE: finished
0926 07:35:34.858179 (+ 103us) write_op.cc:270] Start()
0926 07:35:34.858255 (+ 76us) write_op.cc:276] Timestamp: P: 1790408134858109 usec, L: 0
0926 07:35:34.858269 (+ 14us) op_driver.cc:331] REPLICATION: starting
0926 07:35:34.910234 (+ 51965us) log.cc:842] Serialized 433 byte log entry
0926 07:35:34.921489 (+ 11255us) op_driver.cc:444] REPLICATION: finished
0926 07:35:34.923596 (+ 2107us) write_op.cc:301] APPLY: starting
0926 07:35:34.923644 (+ 48us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:34.923733 (+ 89us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:34.923819 (+ 86us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:34.929861 (+ 6042us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:34.930391 (+ 530us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 07:35:34.930429 (+ 38us) write_op.cc:312] APPLY: finished
0926 07:35:34.930592 (+ 163us) log.cc:842] Serialized 29 byte log entry
0926 07:35:34.930708 (+ 116us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:34.930762 (+ 54us) write_op.cc:454] Released schema lock
0926 07:35:34.930788 (+ 26us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":642304,"child_traces":[["op",{"apply.queue_time_us":1977,"num_ops":2,"prepare.queue_time_us":661005,"prepare.run_cpu_time_us":2264,"prepare.run_wall_time_us":56694,"raft.queue_time_us":76,"raft.run_cpu_time_us":545,"raft.run_wall_time_us":546,"replication_time_us":63157,"threads_started":1}]]}
I20260926 07:35:34.933997 12293 raft_consensus.cc:504] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:34.943627 11754 raft_consensus.cc:348] T cef2e44be4084fc78e61cf822659b4ea P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:34.844183 11116 rpcz_store.cc:276] 0926 07:35:33.659258 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:33.726770 (+ 67512us) service_pool.cc:227] Handling call
0926 07:35:34.841070 (+1114300us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:34.434610 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:34.434959 (+ 349us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:34.434981 (+ 22us) write_op.cc:435] Acquired schema lock
0926 07:35:34.434993 (+ 12us) tablet.cc:660] Decoding operations
0926 07:35:34.435271 (+ 278us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:34.435308 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:34.435328 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0926 07:35:34.435498 (+ 170us) tablet.cc:699] Row locks acquired
0926 07:35:34.435511 (+ 13us) write_op.cc:260] PREPARE: finished
0926 07:35:34.435611 (+ 100us) write_op.cc:270] Start()
0926 07:35:34.435693 (+ 82us) write_op.cc:276] Timestamp: P: 1790408134435552 usec, L: 0
0926 07:35:34.435707 (+ 14us) op_driver.cc:331] REPLICATION: starting
0926 07:35:34.569533 (+133826us) log.cc:842] Serialized 625 byte log entry
0926 07:35:34.578476 (+ 8943us) op_driver.cc:444] REPLICATION: finished
0926 07:35:34.579040 (+ 564us) write_op.cc:301] APPLY: starting
0926 07:35:34.579103 (+ 63us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:34.579237 (+ 134us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:34.579362 (+ 125us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:34.579775 (+ 413us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:34.580045 (+ 270us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:34.580105 (+ 60us) write_op.cc:312] APPLY: finished
0926 07:35:34.580336 (+ 231us) log.cc:842] Serialized 29 byte log entry
0926 07:35:34.580511 (+ 175us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:34.580594 (+ 83us) write_op.cc:454] Released schema lock
0926 07:35:34.580633 (+ 39us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:34.580827 (+ 194us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:34.580859 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":398,"apply.run_cpu_time_us":2013,"apply.run_wall_time_us":2012,"num_ops":2,"prepare.queue_time_us":689005,"prepare.run_cpu_time_us":1991,"prepare.run_wall_time_us":191826,"raft.queue_time_us":108,"raft.run_cpu_time_us":773,"raft.run_wall_time_us":773,"replication_time_us":142679}]]}
I20260926 07:35:34.956889 12285 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249: acquiring CA information for follower catalog manager: success
I20260926 07:35:34.969067 12311 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260926 07:35:34.969240 12293 leader_election.cc:288] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
W20260926 07:35:34.969477 11436 consensus_meta.cc:287] T 2dd5175f0ca941f8b3856ea415e2ee68 P 22f3902d01584936b20437606cb86dd1: Time spent flushing consensus metadata: real 0.562s user 0.011s sys 0.013s
I20260926 07:35:34.974036 11515 heartbeater.cc:464] Registering TS with master...
W20260926 07:35:34.974440 11981 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.61:38823, user_credentials={real_user=slave}} blocked reactor thread for 90187.2us
I20260926 07:35:34.974602 11444 tablet_service.cc:1514] Processing DeleteTablet for tablet c092aff8f2a041ccb09bc4b48e0f359c with delete_type TABLET_DATA_DELETED (replaced by tablet 191f35545a13402db05b71dafa589845) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:34.975452 11444 tablet_service.cc:1514] Processing DeleteTablet for tablet f4aa8bef4bc64953ae2c573238838469 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:33 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:34.977597 11435 tablet_service.cc:1467] Processing CreateTablet for tablet e219b112c1a24bca84befe262adfee19 (DEFAULT_TABLE table=default.Table_d605de609d19450cbe6cf9af7b9774fd [id=d14bc02fec09496db9a0d56d739b5208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:34.978358 11276 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:41350 (request call id 78) took 1045 ms. Trace:
I20260926 07:35:34.978789 11276 rpcz_store.cc:276] 0926 07:35:33.703998 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:33.707455 (+ 3457us) service_pool.cc:227] Handling call
0926 07:35:34.145873 (+438418us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b5400151b54. stack: 00007fedc0a39033 00007fedc0a38f30 00007fedc05ba534 00000000004db2b4 00000000004db249 00007fedc7945471 00007fedc79ba268 00007fedc89a66cd 00007fedc724ee04 00007fedc724ed92 00007fedc724ecc1 00007fedc724ec3c 00007fedc724def2 00007fedc576f94c 00007fedc576ed86 00007fedc576e72f
0926 07:35:34.748976 (+603103us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":493258496}
I20260926 07:35:34.982822 12212 raft_consensus.cc:482] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:34.983273 12212 raft_consensus.cc:504] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:34.985576 12259 ts_tablet_manager.cc:1918] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:34.985941 12259 log.cc:1196] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/c81a59361a6e42b7962b3e027c4a087f
I20260926 07:35:34.986683 12259 ts_tablet_manager.cc:1939] T c81a59361a6e42b7962b3e027c4a087f P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
W20260926 07:35:34.987435 11401 consensus_peers.cc:487] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:34.989850 11746 tablet_bootstrap.cc:654] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:34.993022 11754 raft_consensus.cc:374] T cef2e44be4084fc78e61cf822659b4ea P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:34.994658 12212 leader_election.cc:288] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:34.982079 11117 catalog_manager.cc:5495] T 90f62593b6f844fc8a41b9be240101fd P 22f3902d01584936b20437606cb86dd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22f3902d01584936b20437606cb86dd1 (127.10.171.2). New cstate: current_term: 1 leader_uuid: "22f3902d01584936b20437606cb86dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } health_report { overall_health: UNKNOWN } } }
I20260926 07:35:34.959996 11581 tablet_service.cc:1514] Processing DeleteTablet for tablet f4aa8bef4bc64953ae2c573238838469 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:33 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:34.993602 11754 raft_consensus.cc:729] T cef2e44be4084fc78e61cf822659b4ea P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:34.996953 11754 consensus_queue.cc:260] T cef2e44be4084fc78e61cf822659b4ea P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:34.997215 11117 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 0218d2a727f942e48d770878e6bf4a54 (table default.Table_e8b00b73be7c4818adf8d1e74d731ee6 [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b]). Expected version 3 got 0
I20260926 07:35:34.998746 11117 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet ae16af9edff9401589bcf54eeb3e8dfe (table default.Table_f9178d5e3d934f1f9ef267a782316458 [id=0c18461963f444269787b667b3648709]). Expected version 2 got 0
I20260926 07:35:34.998906 11754 ts_tablet_manager.cc:1428] T cef2e44be4084fc78e61cf822659b4ea P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.072s user 0.005s sys 0.000s
I20260926 07:35:34.999470 11117 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 0eae53ce278749208492815b918582f3 (table default.Table_56493b3efdaa43e684afcd3d4b5dffdd [id=f66bffa952604b28bb51299de795ee2a]). Expected version 1 got 0
I20260926 07:35:35.001269 11285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d0ac73eb239499083eead51e0a7e229. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:35.003134 11462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17871fbd15064daba6e80a34a1b221d3" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:35.004688 11296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f3986a19a5044159c89ae6ba72b247c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:35.006186 12309 consensus_queue.cc:1035] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 07:35:35.008713 11402 leader_election.cc:302] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:35.012872 12309 raft_consensus.cc:2781] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260926 07:35:35.014746 11289 consensus_meta.cc:287] T 5f2ef7ebe6e144529cffd699180d5be8 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent flushing consensus metadata: real 0.805s user 0.004s sys 0.002s
I20260926 07:35:35.016973 11436 tablet_service.cc:1467] Processing CreateTablet for tablet 4081354696234f33b51bb3c0d141f7e6 (DEFAULT_TABLE table=default.Table_11baf2ea5fdc431fae1ae6d0dd3cdb2b [id=1c14eed77a104f2ba5e306f94bffd72a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:35.013252 12309 raft_consensus.cc:482] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:35.017882 12309 raft_consensus.cc:3037] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
W20260926 07:35:34.854813 11756 ts_tablet_manager.cc:1462] T 90f62593b6f844fc8a41b9be240101fd P e9bfebb34fa14cd2ad4843fffbe5c54e: Trace:
0926 07:35:26.654326 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:26.656781 (+ 2455us) service_pool.cc:227] Handling call
0926 07:35:26.663233 (+ 6452us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0926 07:35:26.663330 (+ 97us) ts_tablet_manager.cc:676] Creating new metadata...
0926 07:35:27.159195 (+495865us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:27.459959 (+300764us) tablet_replica.cc:205] Creating consensus instance
0926 07:35:27.460481 (+ 522us) inbound_call.cc:173] Queueing success response
0926 07:35:33.353487 (+5893006us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0926 07:35:34.209906 (+856419us) ts_tablet_manager.cc:1429] Starting tablet replica
0926 07:35:34.480351 (+270445us) tablet_replica.cc:259] Starting instrumentation
0926 07:35:34.480874 (+ 523us) tablet_replica.cc:278] Starting consensus
Metrics: {"mutex_wait_us":72460,"raft.queue_time_us":280213,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":158,"tablet-open.queue_time_us":5893066}
I20260926 07:35:35.024852 11756 tablet_bootstrap.cc:492] T 7e52ec7d802a4e90bccef777f77a2a1b P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:35.027606 11290 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:52070 (request call id 150) took 1192 ms. Trace:
I20260926 07:35:35.027985 11290 rpcz_store.cc:276] 0926 07:35:33.704824 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:33.705418 (+ 594us) service_pool.cc:227] Handling call
0926 07:35:33.761498 (+ 56080us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0926 07:35:33.761575 (+ 77us) ts_tablet_manager.cc:676] Creating new metadata...
0926 07:35:34.300802 (+539227us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:34.870593 (+569791us) tablet_replica.cc:205] Creating consensus instance
0926 07:35:34.897320 (+ 26727us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260926 07:35:34.989797 11401 consensus_peers.cc:487] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:35.030128 12212 consensus_queue.cc:1035] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:35.031280 11114 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_4a4f85ee6205445899cac4b93538c3dc" } new_table_name: "default.Table_e9b22035d94d423280e51a4864e4aa84" modify_external_catalogs: true
W20260926 07:35:35.031689 11246 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.10.171.61:38823, user_credentials={real_user=slave}} blocked reactor thread for 49623.4us
I20260926 07:35:35.035384 12285 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 07:35:35.036491 11118 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_da7cb8d741f244f797eb0847d8500737"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:35.037633 11545 leader_election.cc:341] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Not found: Tablet not found: 05f071b4db7847a5adbd25a71cac3ea7
W20260926 07:35:35.037820 11118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_da7cb8d741f244f797eb0847d8500737 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:35:35.040365 11545 leader_election.cc:341] T 17871fbd15064daba6e80a34a1b221d3 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:34.896416 12253 rpcz_store.cc:276] 0926 07:35:32.771534 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:32.771671 (+ 137us) service_pool.cc:227] Handling call
0926 07:35:34.838064 (+2066393us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:32.971410 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0926 07:35:32.971741 (+ 331us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0926 07:35:32.971772 (+ 31us) alter_schema_op.cc:65] Acquired schema lock
0926 07:35:32.971824 (+ 52us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0926 07:35:32.971937 (+ 113us) alter_schema_op.cc:126] START. Timestamp: 1790408132971774
0926 07:35:32.971958 (+ 21us) op_driver.cc:331] REPLICATION: starting
0926 07:35:32.987743 (+ 15785us) log.cc:842] Serialized 173 byte log entry
0926 07:35:33.614623 (+626880us) op_driver.cc:444] REPLICATION: finished
0926 07:35:33.725365 (+110742us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0926 07:35:33.915300 (+189935us) log.cc:842] Serialized 5 byte log entry
0926 07:35:34.503502 (+588202us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:34.511319 (+ 7817us) log.cc:842] Serialized 14 byte log entry
0926 07:35:34.608991 (+ 97672us) alter_schema_op.cc:71] Released schema lock
0926 07:35:34.744662 (+135671us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":110560,"prepare.queue_time_us":198914,"prepare.run_cpu_time_us":2194,"prepare.run_wall_time_us":77709,"raft.queue_time_us":1150,"raft.run_cpu_time_us":768,"raft.run_wall_time_us":768,"replication_time_us":642552,"thread_start_us":368831,"threads_started":3,"wal-append.queue_time_us":18916}]]}
I20260926 07:35:35.042686 11462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c81a59361a6e42b7962b3e027c4a087f" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:35.044039 11462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90f62593b6f844fc8a41b9be240101fd" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:35.048866 11116 master_service.cc:946] ReplaceTablet: received request to replace tablet 9feaf1ed8ff34928b716fd0b4e41cdde from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:35.055095 11115 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:44836 (request call id 257) took 1190 ms. Trace:
I20260926 07:35:35.055455 11115 rpcz_store.cc:276] 0926 07:35:33.864316 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.175785 (+311469us) service_pool.cc:227] Handling call
0926 07:35:34.345420 (+169635us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812376156 00007fb80feb3034 00007fb80feb2fc2 00007fb80feb2ef1 00007fb80feb2e6c 00007fb80feb2122 00007fb80e15594c 00007fb80e154d86
0926 07:35:34.802085 (+456665us) catalog_manager.cc:2075] Acquired catalog manager lock
0926 07:35:34.874785 (+ 72700us) catalog_manager.cc:2121] Created new table and tablet info
0926 07:35:35.054531 (+179746us) catalog_manager.cc:2181] Wrote table and tablets to system table
0926 07:35:35.054808 (+ 277us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0926 07:35:35.055045 (+ 237us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:34.913491 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:34.913858 (+ 367us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:34.913880 (+ 22us) write_op.cc:435] Acquired schema lock
0926 07:35:34.913893 (+ 13us) tablet.cc:660] Decoding operations
0926 07:35:34.914137 (+ 244us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:34.914171 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:34.914191 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0926 07:35:34.961122 (+ 46931us) tablet.cc:699] Row locks acquired
0926 07:35:34.961147 (+ 25us) write_op.cc:260] PREPARE: finished
0926 07:35:34.961286 (+ 139us) write_op.cc:270] Start()
0926 07:35:34.961400 (+ 114us) write_op.cc:276] Timestamp: P: 1790408134961203 usec, L: 0
0926 07:35:34.961423 (+ 23us) op_driver.cc:331] REPLICATION: starting
0926 07:35:35.002912 (+ 41489us) log.cc:842] Serialized 433 byte log entry
0926 07:35:35.004956 (+ 2044us) spinlock_profiling.cc:243] Waited 41 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032b471 00007fb8103a2198 00007fb810803ade 00007fb8108032b8 00007fb810807ba1 00007fb810807b59 00007fb810807ae9 00007fb810807ab1 00007fb810806dad 0000000000580784 00000000005805b9
0926 07:35:35.051634 (+ 46678us) op_driver.cc:444] REPLICATION: finished
0926 07:35:35.052490 (+ 856us) write_op.cc:301] APPLY: starting
0926 07:35:35.052540 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:35.052635 (+ 95us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:35.052654 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:35.053043 (+ 389us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:35.053182 (+ 139us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 07:35:35.053223 (+ 41us) write_op.cc:312] APPLY: finished
0926 07:35:35.053400 (+ 177us) log.cc:842] Serialized 29 byte log entry
0926 07:35:35.053962 (+ 562us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:35.054038 (+ 76us) write_op.cc:454] Released schema lock
0926 07:35:35.054069 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:35.054240 (+ 171us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:35.054262 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":373100800,"child_traces":[["op",{"apply.queue_time_us":609,"apply.run_cpu_time_us":1526,"apply.run_wall_time_us":2007,"num_ops":2,"prepare.queue_time_us":37749,"prepare.run_cpu_time_us":3393,"prepare.run_wall_time_us":91650,"raft.queue_time_us":2969,"raft.run_cpu_time_us":765,"raft.run_wall_time_us":20559,"replication_time_us":90134,"spinlock_wait_cycles":94312192}]]}
W20260926 07:35:34.873561 11249 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.171.3:45127, user_credentials={real_user=slave}} blocked reactor thread for 124945us
W20260926 07:35:35.062171 11113 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ffb50e0189d64ddc9642037d87aed44c has no consensus state
I20260926 07:35:35.064721 11113 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:44836 (request call id 259) took 1139 ms. Trace:
I20260926 07:35:35.066758 12309 raft_consensus.cc:504] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.069304 11746 tablet_bootstrap.cc:492] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:35.069743 11746 ts_tablet_manager.cc:1397] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.347s user 0.013s sys 0.008s
I20260926 07:35:35.072013 11746 raft_consensus.cc:348] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:35.072935 11746 raft_consensus.cc:374] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:35.073237 11746 raft_consensus.cc:729] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:35.073813 11746 consensus_queue.cc:260] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:35.078785 11608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f20206ce6fd4fd395f1b2b0154245db" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:35.079329 11608 raft_consensus.cc:2454] T 0f20206ce6fd4fd395f1b2b0154245db P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:34.877279 11112 rpcz_store.cc:276] 0926 07:35:33.809117 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:33.810289 (+ 1172us) service_pool.cc:227] Handling call
0926 07:35:33.860160 (+ 49871us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0926 07:35:33.973015 (+112855us) catalog_manager.cc:2752] Modifying in-memory table state
0926 07:35:33.973138 (+ 123us) catalog_manager.cc:2761] Locking tablets
0926 07:35:33.973758 (+ 620us) catalog_manager.cc:2775] Removing table and tablets from system table
0926 07:35:34.682400 (+708642us) catalog_manager.cc:2793] Removing table from by-name map
0926 07:35:34.803307 (+120907us) catalog_manager.cc:2809] Committing in-memory state
0926 07:35:34.803419 (+ 112us) catalog_manager.cc:2813] Aborting table tasks
0926 07:35:34.876826 (+ 73407us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:34.626850 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:34.659867 (+ 33017us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:34.659915 (+ 48us) write_op.cc:435] Acquired schema lock
0926 07:35:34.659931 (+ 16us) tablet.cc:660] Decoding operations
0926 07:35:34.664692 (+ 4761us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:34.664745 (+ 53us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:34.664773 (+ 28us) tablet.cc:683] Acquiring locks for 2 operations
0926 07:35:34.665111 (+ 338us) tablet.cc:699] Row locks acquired
0926 07:35:34.665126 (+ 15us) write_op.cc:260] PREPARE: finished
0926 07:35:34.665253 (+ 127us) write_op.cc:270] Start()
0926 07:35:34.665351 (+ 98us) write_op.cc:276] Timestamp: P: 1790408134665172 usec, L: 0
0926 07:35:34.665366 (+ 15us) op_driver.cc:331] REPLICATION: starting
0926 07:35:34.665874 (+ 508us) log.cc:842] Serialized 728 byte log entry
0926 07:35:34.674674 (+ 8800us) op_driver.cc:444] REPLICATION: finished
0926 07:35:34.675412 (+ 738us) write_op.cc:301] APPLY: starting
0926 07:35:34.675516 (+ 104us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:34.675732 (+ 216us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:34.675869 (+ 137us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:34.681136 (+ 5267us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:34.681288 (+ 152us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0926 07:35:34.681346 (+ 58us) write_op.cc:312] APPLY: finished
0926 07:35:34.681546 (+ 200us) log.cc:842] Serialized 29 byte log entry
0926 07:35:34.681691 (+ 145us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:34.681764 (+ 73us) write_op.cc:454] Released schema lock
0926 07:35:34.681797 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:34.681974 (+ 177us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:34.682007 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":535,"apply.run_cpu_time_us":1955,"apply.run_wall_time_us":6830,"num_ops":2,"prepare.queue_time_us":651392,"prepare.run_cpu_time_us":2715,"prepare.run_wall_time_us":170987,"raft.queue_time_us":187,"raft.run_cpu_time_us":798,"raft.run_wall_time_us":809,"replication_time_us":9242}]]}
I20260926 07:35:35.083519 11607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd4f14abfea4799b75c48531412fc20" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:35.084051 11607 raft_consensus.cc:2454] T dbd4f14abfea4799b75c48531412fc20 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:35.084741 11604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90f62593b6f844fc8a41b9be240101fd" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:35.086059 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f20206ce6fd4fd395f1b2b0154245db" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:35.065068 11113 rpcz_store.cc:276] 0926 07:35:33.925486 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.325220 (+399734us) service_pool.cc:227] Handling call
0926 07:35:34.631631 (+306411us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb8123767d6 00007fb80febc634 00007fb80febc5c2 00007fb80febc4f1 00007fb80febc46c 00007fb80febb722 00007fb80e15594c 00007fb80e154d86
0926 07:35:34.804346 (+172715us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0926 07:35:34.806453 (+ 2107us) catalog_manager.cc:3616] Apply alter schema
0926 07:35:34.887039 (+ 80586us) catalog_manager.cc:3639] Acquired catalog manager lock
0926 07:35:34.887324 (+ 285us) catalog_manager.cc:3786] Updating metadata on disk
0926 07:35:35.061645 (+174321us) catalog_manager.cc:3817] Committing alterations to in-memory state
0926 07:35:35.064670 (+ 3025us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:35.005241 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:35.005670 (+ 429us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:35.005691 (+ 21us) write_op.cc:435] Acquired schema lock
0926 07:35:35.005705 (+ 14us) tablet.cc:660] Decoding operations
0926 07:35:35.005941 (+ 236us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:35.005978 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:35.006011 (+ 33us) tablet.cc:683] Acquiring locks for 1 operations
0926 07:35:35.006147 (+ 136us) tablet.cc:699] Row locks acquired
0926 07:35:35.006160 (+ 13us) write_op.cc:260] PREPARE: finished
0926 07:35:35.006255 (+ 95us) write_op.cc:270] Start()
0926 07:35:35.006327 (+ 72us) write_op.cc:276] Timestamp: P: 1790408135006179 usec, L: 0
0926 07:35:35.006344 (+ 17us) op_driver.cc:331] REPLICATION: starting
0926 07:35:35.006766 (+ 422us) log.cc:842] Serialized 405 byte log entry
0926 07:35:35.059187 (+ 52421us) op_driver.cc:444] REPLICATION: finished
0926 07:35:35.059869 (+ 682us) write_op.cc:301] APPLY: starting
0926 07:35:35.059916 (+ 47us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:35.060007 (+ 91us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:35.060102 (+ 95us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:35.060238 (+ 136us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:35.060347 (+ 109us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:35.060386 (+ 39us) write_op.cc:312] APPLY: finished
0926 07:35:35.060530 (+ 144us) log.cc:842] Serialized 23 byte log entry
0926 07:35:35.061140 (+ 610us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:35.061187 (+ 47us) write_op.cc:454] Released schema lock
0926 07:35:35.061213 (+ 26us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:35.061360 (+ 147us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:35.061385 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":701119744,"child_traces":[["op",{"apply.queue_time_us":555,"apply.run_cpu_time_us":1177,"apply.run_wall_time_us":1653,"num_ops":1,"prepare.queue_time_us":116624,"prepare.run_cpu_time_us":1674,"prepare.run_wall_time_us":45398,"replication_time_us":52770}]]}
I20260926 07:35:35.089742 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd4f14abfea4799b75c48531412fc20" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:35.089986 11316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17871fbd15064daba6e80a34a1b221d3" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
W20260926 07:35:35.091814 11249 leader_election.cc:341] T c81a59361a6e42b7962b3e027c4a087f P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Not found: Tablet not found: c81a59361a6e42b7962b3e027c4a087f
W20260926 07:35:35.100289 11249 consensus_peers.cc:487] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e -> Peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Couldn't send request to peer 6070e9e3d1234a9d83e49431416afade. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9feaf1ed8ff34928b716fd0b4e41cdde. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:35.102231 11288 tablet_service.cc:1514] Processing DeleteTablet for tablet f4aa8bef4bc64953ae2c573238838469 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:33 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:35.106391 11120 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet cef2e44be4084fc78e61cf822659b4ea
I20260926 07:35:35.107700 11120 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:44836 (request call id 260) took 1177 ms. Trace:
I20260926 07:35:35.108805 11746 ts_tablet_manager.cc:1428] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.039s user 0.004s sys 0.002s
W20260926 07:35:35.081537 11401 leader_election.cc:341] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:35.085350 12212 consensus_queue.cc:1035] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:35.085565 12038 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "6070e9e3d1234a9d83e49431416afade" instance_seqno: 1790408119549284) as {username='slave'} at 127.10.171.3:34211; Asking this server to re-register.
I20260926 07:35:35.086131 11402 leader_election.cc:302] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:35.110041 12212 raft_consensus.cc:2781] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:35.110353 12212 raft_consensus.cc:482] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:35.110627 12212 raft_consensus.cc:3037] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:35.107967 11120 rpcz_store.cc:276] 0926 07:35:33.929854 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.423692 (+493838us) service_pool.cc:227] Handling call
0926 07:35:34.626970 (+203278us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812376466 00007fb80feb7b34 00007fb80feb7ac2 00007fb80feb79f1 00007fb80feb796c 00007fb80feb6c22 00007fb80e15594c 00007fb80e154d86
0926 07:35:34.804620 (+177650us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0926 07:35:34.841982 (+ 37362us) catalog_manager.cc:2752] Modifying in-memory table state
0926 07:35:34.842128 (+ 146us) catalog_manager.cc:2761] Locking tablets
0926 07:35:34.842965 (+ 837us) catalog_manager.cc:2775] Removing table and tablets from system table
0926 07:35:35.105975 (+263010us) catalog_manager.cc:2793] Removing table from by-name map
0926 07:35:35.106203 (+ 228us) catalog_manager.cc:2809] Committing in-memory state
0926 07:35:35.106312 (+ 109us) catalog_manager.cc:2813] Aborting table tasks
0926 07:35:35.107649 (+ 1337us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:35.050763 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:35.051122 (+ 359us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:35.051145 (+ 23us) write_op.cc:435] Acquired schema lock
0926 07:35:35.051158 (+ 13us) tablet.cc:660] Decoding operations
0926 07:35:35.068570 (+ 17412us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:35.068615 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:35.068641 (+ 26us) tablet.cc:683] Acquiring locks for 2 operations
0926 07:35:35.068915 (+ 274us) tablet.cc:699] Row locks acquired
0926 07:35:35.068931 (+ 16us) write_op.cc:260] PREPARE: finished
0926 07:35:35.069050 (+ 119us) write_op.cc:270] Start()
0926 07:35:35.069129 (+ 79us) write_op.cc:276] Timestamp: P: 1790408135068968 usec, L: 0
0926 07:35:35.069147 (+ 18us) op_driver.cc:331] REPLICATION: starting
0926 07:35:35.069635 (+ 488us) log.cc:842] Serialized 759 byte log entry
0926 07:35:35.103128 (+ 33493us) op_driver.cc:444] REPLICATION: finished
0926 07:35:35.104220 (+ 1092us) write_op.cc:301] APPLY: starting
0926 07:35:35.104277 (+ 57us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:35.104457 (+ 180us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:35.104564 (+ 107us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:35.104805 (+ 241us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:35.104955 (+ 150us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0926 07:35:35.104993 (+ 38us) write_op.cc:312] APPLY: finished
0926 07:35:35.105153 (+ 160us) log.cc:842] Serialized 29 byte log entry
0926 07:35:35.105712 (+ 559us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:35.105787 (+ 75us) write_op.cc:454] Released schema lock
0926 07:35:35.105814 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:35.105964 (+ 150us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:35.105985 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":464661888,"child_traces":[["op",{"apply.queue_time_us":923,"apply.run_cpu_time_us":1479,"apply.run_wall_time_us":1961,"num_ops":2,"prepare.queue_time_us":160134,"prepare.run_cpu_time_us":2095,"prepare.run_wall_time_us":51243,"raft.queue_time_us":805,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":260,"replication_time_us":33912}]]}
I20260926 07:35:35.112226 11660 heartbeater.cc:464] Registering TS with master...
I20260926 07:35:35.113184 11581 tablet_service.cc:1514] Processing DeleteTablet for tablet cef2e44be4084fc78e61cf822659b4ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:34 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:35.116614 11402 leader_election.cc:302] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:35.117658 12304 raft_consensus.cc:2781] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:35.117944 12304 raft_consensus.cc:482] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:35.118240 12304 raft_consensus.cc:3037] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:35.123063 11436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4081354696234f33b51bb3c0d141f7e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:35.123270 12259 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 153) took 1101 ms. Trace:
I20260926 07:35:35.123564 12259 rpcz_store.cc:276] 0926 07:35:34.022043 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.022974 (+ 931us) service_pool.cc:227] Handling call
0926 07:35:34.231916 (+208942us) ts_tablet_manager.cc:1145] Deleting tablet c81a59361a6e42b7962b3e027c4a087f
0926 07:35:34.231952 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:34.397370 (+165418us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:34.985304 (+587934us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:35.123210 (+137906us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6564,"tablet-delete.queue_time_us":208447}
I20260926 07:35:35.126147 12259 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 155) took 1103 ms. Trace:
I20260926 07:35:35.126741 11437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:40244 (request call id 165) took 1045 ms. Trace:
I20260926 07:35:35.126452 12259 rpcz_store.cc:276] 0926 07:35:34.022651 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.023602 (+ 951us) service_pool.cc:227] Handling call
0926 07:35:35.125895 (+1102293us) ts_tablet_manager.cc:1145] Deleting tablet 509f607b089c44bc9c3f80ed59a68c95
0926 07:35:35.125935 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:35.126096 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1101718}
I20260926 07:35:35.127072 11437 rpcz_store.cc:276] 0926 07:35:34.081567 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.081685 (+ 118us) service_pool.cc:227] Handling call
0926 07:35:34.232341 (+150656us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0926 07:35:34.232472 (+ 131us) ts_tablet_manager.cc:676] Creating new metadata...
0926 07:35:34.754351 (+521879us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:35.126309 (+371958us) tablet_replica.cc:205] Creating consensus instance
0926 07:35:35.126691 (+ 382us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260926 07:35:35.127907 12259 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 157) took 1104 ms. Trace:
I20260926 07:35:35.128903 11435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e219b112c1a24bca84befe262adfee19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:35.129130 11754 tablet_bootstrap.cc:492] T f8f025d645eb478f95fa0aef17a86729 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:35.128856 12259 rpcz_store.cc:276] 0926 07:35:34.023211 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.023539 (+ 328us) service_pool.cc:227] Handling call
0926 07:35:35.127672 (+1104133us) ts_tablet_manager.cc:1145] Deleting tablet 1e1cea4523a64428aac03ca0bdf708c7
0926 07:35:35.127719 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:35.127862 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1102995}
I20260926 07:35:35.129875 12259 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 156) took 1107 ms. Trace:
I20260926 07:35:35.130187 12259 rpcz_store.cc:276] 0926 07:35:34.022958 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.024253 (+ 1295us) service_pool.cc:227] Handling call
0926 07:35:35.129671 (+1105418us) ts_tablet_manager.cc:1145] Deleting tablet 55ad1d96e2314d1eb0a717a3e9ff31bc
0926 07:35:35.129707 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:35.129836 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1104379}
I20260926 07:35:35.131255 12259 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 158) took 1107 ms. Trace:
I20260926 07:35:35.131675 11442 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:40244 (request call id 166) took 1047 ms. Trace:
I20260926 07:35:35.133731 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 124) took 5425 ms. Trace:
I20260926 07:35:35.134073 11871 rpcz_store.cc:276] 0926 07:35:29.708215 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.708333 (+ 118us) service_pool.cc:227] Handling call
0926 07:35:34.552233 (+4843900us) ts_tablet_manager.cc:1145] Deleting tablet 1e1cea4523a64428aac03ca0bdf708c7
0926 07:35:34.552273 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:34.800136 (+247863us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:34.904660 (+104524us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:35.133674 (+229014us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4843381}
I20260926 07:35:35.135860 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 1e1cea4523a64428aac03ca0bdf708c7 (table default.Table_7846bbd3f8cd4fbbb54ce45d2e3ce6b9 [id=41849849f4ae4a9396ccda32d995e21a]) successfully deleted
I20260926 07:35:35.136730 11871 tablet_replica.cc:330] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:35.138170 11871 raft_consensus.cc:2229] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:35.138522 12038 ts_manager.cc:194] Registered new tserver with Master: 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:35.138780 11871 raft_consensus.cc:2258] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:35.140895 11871 ts_tablet_manager.cc:1905] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:35.142395 11606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f581510fe624127a4d27d302f5be614" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:35.142948 11606 raft_consensus.cc:3037] T 3f581510fe624127a4d27d302f5be614 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:35.144870 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet c81a59361a6e42b7962b3e027c4a087f (table default.Table_1b1901dc5bbb4288be18304dd4a9b113 [id=707c7ee5ced14cee8b8005cce80dbbb5]) successfully deleted
I20260926 07:35:35.144999 11112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_e7e2b6923f804b9fac3a16fed6570765"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:35.145653 11105 catalog_manager.cc:4808] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 509f607b089c44bc9c3f80ed59a68c95 because the tablet was not found. No further retry: Not found: Tablet not found: 509f607b089c44bc9c3f80ed59a68c95
I20260926 07:35:35.146262 11290 tablet_service.cc:1514] Processing DeleteTablet for tablet cef2e44be4084fc78e61cf822659b4ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:34 UTC) from {username='slave'} at 127.0.0.1:52070
W20260926 07:35:35.146667 11112 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e7e2b6923f804b9fac3a16fed6570765 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:35:35.146847 11105 catalog_manager.cc:4808] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 1e1cea4523a64428aac03ca0bdf708c7 because the tablet was not found. No further retry: Not found: Tablet not found: 1e1cea4523a64428aac03ca0bdf708c7
W20260926 07:35:35.147616 11105 catalog_manager.cc:4808] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 55ad1d96e2314d1eb0a717a3e9ff31bc because the tablet was not found. No further retry: Not found: Tablet not found: 55ad1d96e2314d1eb0a717a3e9ff31bc
I20260926 07:35:35.136228 12168 ts_tablet_manager.cc:1918] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:35.152400 12168 log.cc:1196] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/04be706330924282961054c5835cec73
I20260926 07:35:35.153294 12168 ts_tablet_manager.cc:1939] T 04be706330924282961054c5835cec73 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:35.154043 12168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 140) took 3161 ms. Trace:
I20260926 07:35:35.154348 12168 rpcz_store.cc:276] 0926 07:35:31.992385 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:31.993092 (+ 707us) service_pool.cc:227] Handling call
0926 07:35:33.327995 (+1334903us) ts_tablet_manager.cc:1145] Deleting tablet 04be706330924282961054c5835cec73
0926 07:35:33.328095 (+ 100us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:34.488583 (+1160488us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:35.072088 (+583505us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:35.153992 (+ 81904us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1333989}
I20260926 07:35:35.155345 11606 raft_consensus.cc:2454] T 3f581510fe624127a4d27d302f5be614 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:35.155872 12168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 141) took 3139 ms. Trace:
W20260926 07:35:35.165035 11546 leader_election.cc:341] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Not found: Tablet not found: 05f071b4db7847a5adbd25a71cac3ea7
I20260926 07:35:35.165391 11546 leader_election.cc:302] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:35.165973 12133 raft_consensus.cc:2726] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260926 07:35:35.166934 11546 leader_election.cc:341] T 17871fbd15064daba6e80a34a1b221d3 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:35.167335 11546 leader_election.cc:302] T 17871fbd15064daba6e80a34a1b221d3 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:35.168083 12133 raft_consensus.cc:2726] T 17871fbd15064daba6e80a34a1b221d3 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:35.170341 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet 04be706330924282961054c5835cec73 (table default.Table_4eb22a54d0df4f808fe85886a5809ab7 [id=cab439115f3c46d089824983fe26d0eb]) successfully deleted
I20260926 07:35:35.171402 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_87b25251764145f7b16c7d779094b2a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:35.175232 11113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_87b25251764145f7b16c7d779094b2a2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:35.156877 11288 tablet_service.cc:1514] Processing DeleteTablet for tablet c092aff8f2a041ccb09bc4b48e0f359c with delete_type TABLET_DATA_DELETED (replaced by tablet 191f35545a13402db05b71dafa589845) from {username='slave'} at 127.0.0.1:52070
W20260926 07:35:35.163219 11247 leader_election.cc:341] T c81a59361a6e42b7962b3e027c4a087f P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: Tablet not RUNNING: SHUTDOWN
I20260926 07:35:35.190537 11754 tablet_bootstrap.cc:654] T f8f025d645eb478f95fa0aef17a86729 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:35.203488 12038 ts_manager.cc:194] Registered new tserver with Master: 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:35.211911 11756 tablet_bootstrap.cc:654] T 7e52ec7d802a4e90bccef777f77a2a1b P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:35.176971 11247 leader_election.cc:302] T c81a59361a6e42b7962b3e027c4a087f P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade
W20260926 07:35:35.218859 11247 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.2:42379, user_credentials={real_user=slave}} blocked reactor thread for 55664.9us
W20260926 07:35:35.225531 11114 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 52d064b684d0491e91f176d24eddabe1 consensus state has no leader
I20260926 07:35:35.219621 11247 leader_election.cc:302] T 90f62593b6f844fc8a41b9be240101fd P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade
I20260926 07:35:35.234738 11754 tablet_bootstrap.cc:492] T f8f025d645eb478f95fa0aef17a86729 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:35.235153 11754 ts_tablet_manager.cc:1397] T f8f025d645eb478f95fa0aef17a86729 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.106s user 0.006s sys 0.005s
I20260926 07:35:35.237303 12305 raft_consensus.cc:2726] T 90f62593b6f844fc8a41b9be240101fd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:35.237920 11754 raft_consensus.cc:348] T f8f025d645eb478f95fa0aef17a86729 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.239960 11754 raft_consensus.cc:374] T f8f025d645eb478f95fa0aef17a86729 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:35.240247 11754 raft_consensus.cc:729] T f8f025d645eb478f95fa0aef17a86729 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:35.240808 11754 consensus_queue.cc:260] T f8f025d645eb478f95fa0aef17a86729 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.242228 11754 ts_tablet_manager.cc:1428] T f8f025d645eb478f95fa0aef17a86729 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260926 07:35:35.249401 12306 raft_consensus.cc:2726] T c81a59361a6e42b7962b3e027c4a087f P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:35.220813 12168 rpcz_store.cc:276] 0926 07:35:32.016530 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:32.016666 (+ 136us) service_pool.cc:227] Handling call
0926 07:35:35.155544 (+3138878us) ts_tablet_manager.cc:1145] Deleting tablet 6d54a5c32da5444ca0a2939461f3a0f6
0926 07:35:35.155587 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:35.155833 (+ 246us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3138333}
I20260926 07:35:35.260757 12168 tablet_replica.cc:330] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:35.261610 12168 raft_consensus.cc:2229] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:35.262480 12168 raft_consensus.cc:2258] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 07:35:35.263139 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 6d54a5c32da5444ca0a2939461f3a0f6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d54a5c32da5444ca0a2939461f3a0f6 already in progress: creating tablet
I20260926 07:35:35.273978 11285 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:52070 (request call id 163) took 1141 ms. Trace:
I20260926 07:35:35.290020 11118 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_286cfdaad687460f8aa8dff237604241"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:35.290723 11114 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_0d113826d96240549bc980febd8776e0" } new_table_name: "default.Table_3a3f3643ccfd4c82a0a33fcaed606b63" modify_external_catalogs: true
W20260926 07:35:35.291388 11118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_286cfdaad687460f8aa8dff237604241 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:35.292738 11754 tablet_bootstrap.cc:492] T 248ab84a1f284c2bbddf62f00b485e13 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:35.297664 11116 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_fd842e14c4984402b2b1b77baa1f9360" } modify_external_catalogs: true
I20260926 07:35:35.297873 11754 tablet_bootstrap.cc:654] T 248ab84a1f284c2bbddf62f00b485e13 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:35.298103 11116 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_fd842e14c4984402b2b1b77baa1f9360" } modify_external_catalogs: true
I20260926 07:35:35.309365 11119 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet 0218d2a727f942e48d770878e6bf4a54 (table default.Table_e8b00b73be7c4818adf8d1e74d731ee6 [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b]). Expected version 3 got 0
I20260926 07:35:35.310078 11119 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet 0eae53ce278749208492815b918582f3 (table default.Table_56493b3efdaa43e684afcd3d4b5dffdd [id=f66bffa952604b28bb51299de795ee2a]). Expected version 1 got 0
W20260926 07:35:35.311764 11107 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 52d064b684d0491e91f176d24eddabe1 (table default.Table_e9b22035d94d423280e51a4864e4aa84 [id=d19e4ba11fa14fa18c7ae93bf40f7c89], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 52d064b684d0491e91f176d24eddabe1 consensus state has no leader
I20260926 07:35:35.318352 11113 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_8d6e8e19741e40d0a01d30a0f21fef8a" } modify_external_catalogs: true
I20260926 07:35:35.318886 11113 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_8d6e8e19741e40d0a01d30a0f21fef8a" } modify_external_catalogs: true
I20260926 07:35:35.323117 11871 ts_tablet_manager.cc:1918] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260926 07:35:35.326630 11114 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 16ab928a487c4f499aba3b366d01d89b consensus state has no leader
I20260926 07:35:35.344164 11871 log.cc:1196] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/a000cb9928de4459be1e6b1b7a81848a
I20260926 07:35:35.344928 11871 ts_tablet_manager.cc:1939] T a000cb9928de4459be1e6b1b7a81848a P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:35.346868 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 126) took 5522 ms. Trace:
I20260926 07:35:35.349784 12133 raft_consensus.cc:482] T cef2e44be4084fc78e61cf822659b4ea P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:35.350241 12133 raft_consensus.cc:504] T cef2e44be4084fc78e61cf822659b4ea P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.352411 12133 leader_election.cc:288] T cef2e44be4084fc78e61cf822659b4ea P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:35.347188 11871 rpcz_store.cc:276] 0926 07:35:29.824144 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:29.824282 (+ 138us) service_pool.cc:227] Handling call
0926 07:35:35.136598 (+5312316us) ts_tablet_manager.cc:1145] Deleting tablet a000cb9928de4459be1e6b1b7a81848a
0926 07:35:35.136639 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:35.244727 (+108088us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:35.322902 (+ 78175us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:35.346817 (+ 23915us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5311744}
I20260926 07:35:35.131599 12259 rpcz_store.cc:276] 0926 07:35:34.023518 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.025450 (+ 1932us) service_pool.cc:227] Handling call
0926 07:35:35.131007 (+1105557us) ts_tablet_manager.cc:1145] Deleting tablet d6fc6178f1e54c3984a20873864233cd
0926 07:35:35.131048 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:35.131203 (+ 155us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1105086}
I20260926 07:35:35.380810 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet a000cb9928de4459be1e6b1b7a81848a (table default.Table_d0380e9d3f26439bbc91cf476aeae69f [id=19875953f2b6484f934503404af57593]) successfully deleted
I20260926 07:35:35.093986 12309 leader_election.cc:288] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:35.156361 11402 leader_election.cc:302] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:35.381681 11871 tablet_replica.cc:330] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
W20260926 07:35:35.381544 11402 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.3:45127, user_credentials={real_user=slave}} blocked reactor thread for 225240us
I20260926 07:35:35.381673 12259 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 159) took 1349 ms. Trace:
I20260926 07:35:35.382016 12259 rpcz_store.cc:276] 0926 07:35:34.032185 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.032282 (+ 97us) service_pool.cc:227] Handling call
0926 07:35:35.381388 (+1349106us) ts_tablet_manager.cc:1145] Deleting tablet a0bdf6a4656e4217b0a7013fddc4cca8
0926 07:35:35.381430 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:35.381622 (+ 192us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1348576}
I20260926 07:35:35.383014 12259 tablet_replica.cc:330] T a8b629a4aba045b697e33bbebecb05b7 P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
W20260926 07:35:35.163708 11401 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.1:41829, user_credentials={real_user=slave}} blocked reactor thread for 82195.7us
W20260926 07:35:35.385020 11401 consensus_peers.cc:487] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260926 07:35:35.385526 11401 leader_election.cc:341] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
W20260926 07:35:35.386227 11401 leader_election.cc:341] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:35.386438 12259 raft_consensus.cc:2229] T a8b629a4aba045b697e33bbebecb05b7 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:35.387038 12259 raft_consensus.cc:2258] T a8b629a4aba045b697e33bbebecb05b7 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:35.387096 12293 raft_consensus.cc:2781] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:35.387497 12293 raft_consensus.cc:686] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
W20260926 07:35:35.387743 11105 catalog_manager.cc:4808] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet d6fc6178f1e54c3984a20873864233cd because the tablet was not found. No further retry: Not found: Tablet not found: d6fc6178f1e54c3984a20873864233cd
I20260926 07:35:35.388247 12214 raft_consensus.cc:482] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:35.388223 12293 consensus_queue.cc:237] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.388664 12214 raft_consensus.cc:504] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:35.389340 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef2e44be4084fc78e61cf822659b4ea" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:35.389534 11754 tablet_bootstrap.cc:492] T 248ab84a1f284c2bbddf62f00b485e13 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:35.389912 11754 ts_tablet_manager.cc:1397] T 248ab84a1f284c2bbddf62f00b485e13 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.097s user 0.008s sys 0.000s
I20260926 07:35:35.390386 12214 leader_election.cc:288] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:35.166721 12304 raft_consensus.cc:504] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.392467 12304 leader_election.cc:288] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:35.188561 11746 tablet_bootstrap.cc:492] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:35.393289 12259 ts_tablet_manager.cc:1905] T a8b629a4aba045b697e33bbebecb05b7 P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:35.176573 11439 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:40244 (request call id 152) took 1170 ms. Trace:
I20260926 07:35:35.187803 12212 raft_consensus.cc:504] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.396993 11439 rpcz_store.cc:276] 0926 07:35:34.006065 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.012207 (+ 6142us) service_pool.cc:227] Handling call
0926 07:35:34.231231 (+219024us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0926 07:35:34.231319 (+ 88us) ts_tablet_manager.cc:676] Creating new metadata...
0926 07:35:35.022310 (+790991us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:35.097218 (+ 74908us) tablet_replica.cc:205] Creating consensus instance
0926 07:35:35.176514 (+ 79296us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260926 07:35:35.398839 12212 leader_election.cc:288] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:35.357055 11437 tablet_service.cc:1514] Processing DeleteTablet for tablet cef2e44be4084fc78e61cf822659b4ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:34 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:35.402263 11442 rpcz_store.cc:276] 0926 07:35:34.084065 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.084259 (+ 194us) service_pool.cc:227] Handling call
0926 07:35:34.232674 (+148415us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0926 07:35:34.232767 (+ 93us) ts_tablet_manager.cc:676] Creating new metadata...
0926 07:35:35.074088 (+841321us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:35.131264 (+ 57176us) tablet_replica.cc:205] Creating consensus instance
0926 07:35:35.131625 (+ 361us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260926 07:35:35.408332 11105 catalog_manager.cc:4808] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet a0bdf6a4656e4217b0a7013fddc4cca8 because the tablet was not found. No further retry: Not found: Tablet not found: a0bdf6a4656e4217b0a7013fddc4cca8
I20260926 07:35:35.409783 11285 rpcz_store.cc:276] 0926 07:35:34.132463 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.155899 (+ 23436us) service_pool.cc:227] Handling call
0926 07:35:35.000732 (+844833us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b5800001898. stack: 00007fedc0a39033 00007fedc0a38f30 00007fedc05ba534 00000000004db2b4 00000000004db249 00007fedc08aff11 00007fedc0a33851 00007fedc0a33702 00007fedc98cab09 00007fedc8927730 00007fedc8a09d21 00007fedc89e71e3 00007fedc8998869 00007fedc708ac44 00007fedc708abd2 00007fedc708ab01
0926 07:35:35.000848 (+ 116us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0926 07:35:35.000922 (+ 74us) ts_tablet_manager.cc:676] Creating new metadata...
0926 07:35:35.159188 (+158266us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:35.273570 (+114382us) tablet_replica.cc:205] Creating consensus instance
0926 07:35:35.273930 (+ 360us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":40486144}
I20260926 07:35:35.298573 11296 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:52070 (request call id 164) took 1134 ms. Trace:
I20260926 07:35:35.353770 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef2e44be4084fc78e61cf822659b4ea" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
W20260926 07:35:35.411233 11545 leader_election.cc:341] T cef2e44be4084fc78e61cf822659b4ea P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:35.411497 11296 rpcz_store.cc:276] 0926 07:35:34.164280 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.330894 (+166614us) service_pool.cc:227] Handling call
0926 07:35:35.004219 (+673325us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b5800001898. stack: 00007fedc0a39033 00007fedc0a38f30 00007fedc05ba534 00000000004db2b4 00000000004db249 00007fedc08aff11 00007fedc0a33851 00007fedc0a33e82 00007fedc98f38b9 00007fedc89275d8 00007fedc89e72a0 00007fedc8998869 00007fedc708ac44 00007fedc708abd2 00007fedc708ab01 00007fedc708aa7c
0926 07:35:35.004271 (+ 52us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0926 07:35:35.004344 (+ 73us) ts_tablet_manager.cc:676] Creating new metadata...
0926 07:35:35.172147 (+167803us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:35.298009 (+125862us) tablet_replica.cc:205] Creating consensus instance
0926 07:35:35.298420 (+ 411us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":38213376}
I20260926 07:35:35.413455 11607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd4f14abfea4799b75c48531412fc20" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:35.412467 11754 raft_consensus.cc:348] T 248ab84a1f284c2bbddf62f00b485e13 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:35.356194 12168 ts_tablet_manager.cc:1905] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:35.396924 11314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f581510fe624127a4d27d302f5be614" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:35.413936 11607 raft_consensus.cc:3037] T dbd4f14abfea4799b75c48531412fc20 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
W20260926 07:35:35.413946 11546 leader_election.cc:341] T cef2e44be4084fc78e61cf822659b4ea P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:35.417128 11546 leader_election.cc:302] T cef2e44be4084fc78e61cf822659b4ea P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:35.417800 11606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f20206ce6fd4fd395f1b2b0154245db" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:35.418038 12296 raft_consensus.cc:2726] T cef2e44be4084fc78e61cf822659b4ea P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:35.418380 11606 raft_consensus.cc:3037] T 0f20206ce6fd4fd395f1b2b0154245db P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:35.413936 11754 raft_consensus.cc:374] T 248ab84a1f284c2bbddf62f00b485e13 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:35.420517 11754 raft_consensus.cc:729] T 248ab84a1f284c2bbddf62f00b485e13 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:35.421111 11314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248ab84a1f284c2bbddf62f00b485e13" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:35.421176 11754 consensus_queue.cc:260] T 248ab84a1f284c2bbddf62f00b485e13 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:35.422067 11401 leader_election.cc:341] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:35.422497 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd4f14abfea4799b75c48531412fc20" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:35.424072 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f20206ce6fd4fd395f1b2b0154245db" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:35.424671 11746 tablet_bootstrap.cc:654] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:35.425866 11401 leader_election.cc:341] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
W20260926 07:35:35.427482 11401 leader_election.cc:341] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
W20260926 07:35:35.428046 11401 leader_election.cc:341] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:35.429117 11118 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_ba08930510174e65840452201a15f719" } new_table_name: "default.Table_8439aa0b11574c5b822c46f01f75d842" modify_external_catalogs: true
I20260926 07:35:35.436247 11756 tablet_bootstrap.cc:492] T 7e52ec7d802a4e90bccef777f77a2a1b P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:35.436620 11756 ts_tablet_manager.cc:1397] T 7e52ec7d802a4e90bccef777f77a2a1b P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.412s user 0.009s sys 0.005s
I20260926 07:35:35.445712 11114 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_a3b47c514b814ebba363e6a1fa5c1743"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:35.446413 11607 raft_consensus.cc:2454] T dbd4f14abfea4799b75c48531412fc20 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
W20260926 07:35:35.447140 11114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a3b47c514b814ebba363e6a1fa5c1743 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:35.448029 11871 raft_consensus.cc:2229] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:35.448561 11606 raft_consensus.cc:2454] T 0f20206ce6fd4fd395f1b2b0154245db P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:35.448904 11871 raft_consensus.cc:2258] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 07:35:35.449606 11118 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 248ab84a1f284c2bbddf62f00b485e13 consensus state has no leader
I20260926 07:35:35.450933 11871 ts_tablet_manager.cc:1905] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:35.452260 11402 leader_election.cc:302] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters: e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:35.454825 12306 consensus_queue.cc:1035] T 9feaf1ed8ff34928b716fd0b4e41cdde P e9bfebb34fa14cd2ad4843fffbe5c54e [LEADER]: Connected to new peer: Peer: permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:35.424183 11754 ts_tablet_manager.cc:1428] T 248ab84a1f284c2bbddf62f00b485e13 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.034s user 0.006s sys 0.001s
I20260926 07:35:35.464818 12292 raft_consensus.cc:2781] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:35.465224 12292 raft_consensus.cc:686] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:35.465659 11754 tablet_bootstrap.cc:492] T b8e926ee35854c28a8c698384526fe38 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:35.465942 12292 consensus_queue.cc:237] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.477821 11402 leader_election.cc:302] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters: e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:35.478751 12212 raft_consensus.cc:2781] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:35.479151 12212 raft_consensus.cc:686] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:35.479867 12212 consensus_queue.cc:237] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.492233 11754 tablet_bootstrap.cc:654] T b8e926ee35854c28a8c698384526fe38 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:35.496738 11746 tablet_bootstrap.cc:492] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:35.497177 11746 ts_tablet_manager.cc:1397] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.309s user 0.011s sys 0.000s
I20260926 07:35:35.499081 11114 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_6fd1abf4390246fa9ab2f95d2c4e76c3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:35.500460 11114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6fd1abf4390246fa9ab2f95d2c4e76c3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:35.514585 11118 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_3748c2123ce44306a4738ddfd5df4847"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:35.516047 11118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3748c2123ce44306a4738ddfd5df4847 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:35.533026 11115 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_72783eb75b984ed394613e1333b94227" } new_table_name: "default.Table_9158764a5a0d463ebc8413d234878ee7" modify_external_catalogs: true
I20260926 07:35:35.537398 11746 raft_consensus.cc:348] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.538041 11746 raft_consensus.cc:374] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:35.538267 11746 raft_consensus.cc:729] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:35.538880 11746 consensus_queue.cc:260] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.540241 11746 ts_tablet_manager.cc:1428] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.043s user 0.004s sys 0.001s
I20260926 07:35:35.540954 11746 tablet_bootstrap.cc:492] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:35.500146 11604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248ab84a1f284c2bbddf62f00b485e13" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:35.550705 11604 raft_consensus.cc:2454] T 248ab84a1f284c2bbddf62f00b485e13 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:35.551767 11402 leader_election.cc:302] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters: e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:35.552884 12292 raft_consensus.cc:482] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:35.553323 12292 raft_consensus.cc:504] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.555747 12292 leader_election.cc:288] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:35.556247 12292 raft_consensus.cc:2781] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:35.556591 12292 raft_consensus.cc:482] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:35.556978 12292 raft_consensus.cc:3037] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:35.562935 11604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cd482fb159d418291cb02bc9138b076" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:35.563627 11604 raft_consensus.cc:2454] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:35.568991 11402 leader_election.cc:302] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:35.569653 12291 raft_consensus.cc:2781] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:34.382230 12071 rpcz_store.cc:276] 0926 07:35:31.711529 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:31.711719 (+ 190us) service_pool.cc:227] Handling call
0926 07:35:32.990881 (+1279162us) raft_consensus.cc:1457] Updating replica for 200 ops
0926 07:35:33.291473 (+300592us) raft_consensus.cc:1520] Early marking committed up to index 5
0926 07:35:33.545504 (+254031us) raft_consensus.cc:1525] Triggering prepare for 200 ops
0926 07:35:33.546178 (+ 674us) write_op.cc:270] Start()
0926 07:35:33.546330 (+ 152us) write_op.cc:276] Timestamp: P: 1790408121545664 usec, L: 0
0926 07:35:33.560481 (+ 14151us) write_op.cc:270] Start()
0926 07:35:33.560586 (+ 105us) write_op.cc:276] Timestamp: P: 1790408121548278 usec, L: 0
0926 07:35:33.561870 (+ 1284us) write_op.cc:270] Start()
0926 07:35:33.561975 (+ 105us) write_op.cc:276] Timestamp: P: 1790408121551474 usec, L: 0
0926 07:35:33.562518 (+ 543us) write_op.cc:270] Start()
0926 07:35:33.562644 (+ 126us) write_op.cc:276] Timestamp: P: 1790408121570701 usec, L: 0
0926 07:35:33.563184 (+ 540us) write_op.cc:270] Start()
0926 07:35:33.563309 (+ 125us) write_op.cc:276] Timestamp: P: 1790408121602566 usec, L: 0
0926 07:35:33.563818 (+ 509us) write_op.cc:270] Start()
0926 07:35:33.563953 (+ 135us) write_op.cc:276] Timestamp: P: 1790408121605530 usec, L: 0
0926 07:35:33.564477 (+ 524us) write_op.cc:270] Start()
0926 07:35:33.564798 (+ 321us) write_op.cc:276] Timestamp: P: 1790408121608442 usec, L: 0
0926 07:35:33.565390 (+ 592us) write_op.cc:270] Start()
0926 07:35:33.565508 (+ 118us) write_op.cc:276] Timestamp: P: 1790408121610457 usec, L: 0
0926 07:35:33.566034 (+ 526us) write_op.cc:270] Start()
0926 07:35:33.566131 (+ 97us) write_op.cc:276] Timestamp: P: 1790408121612472 usec, L: 0
0926 07:35:33.566633 (+ 502us) write_op.cc:270] Start()
0926 07:35:33.566721 (+ 88us) write_op.cc:276] Timestamp: P: 1790408121663989 usec, L: 0
0926 07:35:33.567220 (+ 499us) write_op.cc:270] Start()
0926 07:35:33.567333 (+ 113us) write_op.cc:276] Timestamp: P: 1790408121821869 usec, L: 0
0926 07:35:33.567815 (+ 482us) write_op.cc:270] Start()
0926 07:35:33.567911 (+ 96us) write_op.cc:276] Timestamp: P: 1790408121835703 usec, L: 0
0926 07:35:33.577156 (+ 9245us) write_op.cc:270] Start()
0926 07:35:33.577243 (+ 87us) write_op.cc:276] Timestamp: P: 1790408121840174 usec, L: 0
0926 07:35:33.577692 (+ 449us) write_op.cc:270] Start()
0926 07:35:33.577769 (+ 77us) write_op.cc:276] Timestamp: P: 1790408121851747 usec, L: 0
0926 07:35:33.578173 (+ 404us) write_op.cc:270] Start()
0926 07:35:33.578247 (+ 74us) write_op.cc:276] Timestamp: P: 1790408121873334 usec, L: 0
0926 07:35:33.578642 (+ 395us) write_op.cc:270] Start()
0926 07:35:33.578721 (+ 79us) write_op.cc:276] Timestamp: P: 1790408121966541 usec, L: 0
0926 07:35:33.579156 (+ 435us) write_op.cc:270] Start()
0926 07:35:33.579229 (+ 73us) write_op.cc:276] Timestamp: P: 1790408122048389 usec, L: 0
0926 07:35:33.579640 (+ 411us) write_op.cc:270] Start()
0926 07:35:33.579705 (+ 65us) write_op.cc:276] Timestamp: P: 1790408122111813 usec, L: 0
0926 07:35:33.580088 (+ 383us) write_op.cc:270] Start()
0926 07:35:33.584184 (+ 4096us) write_op.cc:276] Timestamp: P: 1790408122115890 usec, L: 0
0926 07:35:33.584751 (+ 567us) write_op.cc:270] Start()
0926 07:35:33.584849 (+ 98us) write_op.cc:276] Timestamp: P: 1790408122145445 usec, L: 0
0926 07:35:33.585332 (+ 483us) write_op.cc:270] Start()
0926 07:35:33.585432 (+ 100us) write_op.cc:276] Timestamp: P: 1790408122157524 usec, L: 0
0926 07:35:33.585936 (+ 504us) write_op.cc:270] Start()
0926 07:35:33.586019 (+ 83us) write_op.cc:276] Timestamp: P: 1790408122159618 usec, L: 0
0926 07:35:33.586489 (+ 470us) write_op.cc:270] Start()
0926 07:35:33.586641 (+ 152us) write_op.cc:276] Timestamp: P: 1790408122172333 usec, L: 0
0926 07:35:33.587136 (+ 495us) write_op.cc:270] Start()
0926 07:35:33.587222 (+ 86us) write_op.cc:276] Timestamp: P: 1790408122174527 usec, L: 0
0926 07:35:33.587849 (+ 627us) write_op.cc:270] Start()
0926 07:35:33.587940 (+ 91us) write_op.cc:276] Timestamp: P: 1790408122268954 usec, L: 0
0926 07:35:33.593273 (+ 5333us) write_op.cc:270] Start()
0926 07:35:33.593345 (+ 72us) write_op.cc:276] Timestamp: P: 1790408122306563 usec, L: 0
0926 07:35:33.593754 (+ 409us) write_op.cc:270] Start()
0926 07:35:33.593824 (+ 70us) write_op.cc:276] Timestamp: P: 1790408122323339 usec, L: 0
0926 07:35:33.594209 (+ 385us) write_op.cc:270] Start()
0926 07:35:33.594279 (+ 70us) write_op.cc:276] Timestamp: P: 1790408122371430 usec, L: 0
0926 07:35:33.594664 (+ 385us) write_op.cc:270] Start()
0926 07:35:33.594758 (+ 94us) write_op.cc:276] Timestamp: P: 1790408122401515 usec, L: 0
0926 07:35:33.595170 (+ 412us) write_op.cc:270] Start()
0926 07:35:33.595240 (+ 70us) write_op.cc:276] Timestamp: P: 1790408122430353 usec, L: 0
0926 07:35:33.595634 (+ 394us) write_op.cc:270] Start()
0926 07:35:33.595706 (+ 72us) write_op.cc:276] Timestamp: P: 1790408122463030 usec, L: 0
0926 07:35:33.596080 (+ 374us) write_op.cc:270] Start()
0926 07:35:33.609685 (+ 13605us) write_op.cc:276] Timestamp: P: 1790408122530539 usec, L: 0
0926 07:35:33.610208 (+ 523us) write_op.cc:270] Start()
0926 07:35:33.610282 (+ 74us) write_op.cc:276] Timestamp: P: 1790408122534143 usec, L: 0
0926 07:35:33.610707 (+ 425us) write_op.cc:270] Start()
0926 07:35:33.610776 (+ 69us) write_op.cc:276] Timestamp: P: 1790408122536657 usec, L: 0
0926 07:35:33.611171 (+ 395us) write_op.cc:270] Start()
0926 07:35:33.611239 (+ 68us) write_op.cc:276] Timestamp: P: 1790408122545759 usec, L: 0
0926 07:35:33.611645 (+ 406us) write_op.cc:270] Start()
0926 07:35:33.611728 (+ 83us) write_op.cc:276] Timestamp: P: 1790408122628199 usec, L: 0
0926 07:35:33.612118 (+ 390us) write_op.cc:270] Start()
0926 07:35:33.612201 (+ 83us) write_op.cc:276] Timestamp: P: 1790408122631132 usec, L: 0
0926 07:35:33.612622 (+ 421us) write_op.cc:270] Start()
0926 07:35:33.612693 (+ 71us) write_op.cc:276] Timestamp: P: 1790408122664718 usec, L: 0
0926 07:35:33.613080 (+ 387us) write_op.cc:270] Start()
0926 07:35:33.613155 (+ 75us) write_op.cc:276] Timestamp: P: 1790408122705651 usec, L: 0
0926 07:35:33.613559 (+ 404us) write_op.cc:270] Start()
0926 07:35:33.613631 (+ 72us) write_op.cc:276] Timestamp: P: 1790408122711199 usec, L: 0
0926 07:35:33.614033 (+ 402us) write_op.cc:270] Start()
0926 07:35:33.614109 (+ 76us) write_op.cc:276] Timestamp: P: 1790408122773035 usec, L: 0
0926 07:35:33.614510 (+ 401us) write_op.cc:270] Start()
0926 07:35:33.614578 (+ 68us) write_op.cc:276] Timestamp: P: 1790408122789056 usec, L: 0
0926 07:35:33.614955 (+ 377us) write_op.cc:270] Start()
0926 07:35:33.615026 (+ 71us) write_op.cc:276] Timestamp: P: 1790408122793224 usec, L: 0
0926 07:35:33.615424 (+ 398us) write_op.cc:270] Start()
0926 07:35:33.615505 (+ 81us) write_op.cc:276] Timestamp: P: 1790408122809803 usec, L: 0
0926 07:35:33.615889 (+ 384us) write_op.cc:270] Start()
0926 07:35:33.615955 (+ 66us) write_op.cc:276] Timestamp: P: 1790408122906897 usec, L: 0
0926 07:35:33.632137 (+ 16182us) write_op.cc:270] Start()
0926 07:35:33.632229 (+ 92us) write_op.cc:276] Timestamp: P: 1790408122920308 usec, L: 0
0926 07:35:33.632649 (+ 420us) write_op.cc:270] Start()
0926 07:35:33.632808 (+ 159us) write_op.cc:276] Timestamp: P: 1790408122994052 usec, L: 0
0926 07:35:33.633275 (+ 467us) write_op.cc:270] Start()
0926 07:35:33.633344 (+ 69us) write_op.cc:276] Timestamp: P: 1790408122999156 usec, L: 0
0926 07:35:33.633746 (+ 402us) write_op.cc:270] Start()
0926 07:35:33.633818 (+ 72us) write_op.cc:276] Timestamp: P: 1790408123009308 usec, L: 0
0926 07:35:33.634212 (+ 394us) write_op.cc:270] Start()
0926 07:35:33.634281 (+ 69us) write_op.cc:276] Timestamp: P: 1790408123047384 usec, L: 0
0926 07:35:33.634717 (+ 436us) write_op.cc:270] Start()
0926 07:35:33.634783 (+ 66us) write_op.cc:276] Timestamp: P: 1790408123050914 usec, L: 0
0926 07:35:33.635182 (+ 399us) write_op.cc:270] Start()
0926 07:35:33.635260 (+ 78us) write_op.cc:276] Timestamp: P: 1790408123052845 usec, L: 0
0926 07:35:33.635693 (+ 433us) write_op.cc:270] Start()
0926 07:35:33.635766 (+ 73us) write_op.cc:276] Timestamp: P: 1790408123078953 usec, L: 0
0926 07:35:33.636183 (+ 417us) write_op.cc:270] Start()
0926 07:35:33.636255 (+ 72us) write_op.cc:276] Timestamp: P: 1790408123099776 usec, L: 0
0926 07:35:33.636660 (+ 405us) write_op.cc:270] Start()
0926 07:35:33.636733 (+ 73us) write_op.cc:276] Timestamp: P: 1790408123142302 usec, L: 0
0926 07:35:33.637134 (+ 401us) write_op.cc:270] Start()
0926 07:35:33.637214 (+ 80us) write_op.cc:276] Timestamp: P: 1790408123149398 usec, L: 0
0926 07:35:33.645764 (+ 8550us) write_op.cc:270] Start()
0926 07:35:33.645865 (+ 101us) write_op.cc:276] Timestamp: P: 1790408123176664 usec, L: 0
0926 07:35:33.646429 (+ 564us) write_op.cc:270] Start()
0926 07:35:33.646523 (+ 94us) write_op.cc:276] Timestamp: P: 1790408123213806 usec, L: 0
0926 07:35:33.648717 (+ 2194us) write_op.cc:270] Start()
0926 07:35:33.648814 (+ 97us) write_op.cc:276] Timestamp: P: 1790408123258936 usec, L: 0
0926 07:35:33.651514 (+ 2700us) write_op.cc:270] Start()
0926 07:35:33.651727 (+ 213us) write_op.cc:276] Timestamp: P: 1790408123292517 usec, L: 0
0926 07:35:33.652377 (+ 650us) write_op.cc:270] Start()
0926 07:35:33.652482 (+ 105us) write_op.cc:276] Timestamp: P: 1790408123294436 usec, L: 0
0926 07:35:33.653013 (+ 531us) write_op.cc:270] Start()
0926 07:35:33.653110 (+ 97us) write_op.cc:276] Timestamp: P: 1790408123352000 usec, L: 0
0926 07:35:33.653604 (+ 494us) write_op.cc:270] Start()
0926 07:35:33.653698 (+ 94us) write_op.cc:276] Timestamp: P: 1790408123393280 usec, L: 0
0926 07:35:33.654171 (+ 473us) write_op.cc:270] Start()
0926 07:35:33.654259 (+ 88us) write_op.cc:276] Timestamp: P: 1790408123438811 usec, L: 0
0926 07:35:33.654832 (+ 573us) write_op.cc:270] Start()
0926 07:35:33.655158 (+ 326us) write_op.cc:276] Timestamp: P: 1790408123513090 usec, L: 0
0926 07:35:33.655590 (+ 432us) write_op.cc:270] Start()
0926 07:35:33.655659 (+ 69us) write_op.cc:276] Timestamp: P: 1790408123548795 usec, L: 0
0926 07:35:33.656063 (+ 404us) write_op.cc:270] Start()
0926 07:35:33.656169 (+ 106us) write_op.cc:276] Timestamp: P: 1790408123570247 usec, L: 0
0926 07:35:33.656581 (+ 412us) write_op.cc:270] Start()
0926 07:35:33.656650 (+ 69us) write_op.cc:276] Timestamp: P: 1790408123601166 usec, L: 0
0926 07:35:33.657098 (+ 448us) write_op.cc:270] Start()
0926 07:35:33.657174 (+ 76us) write_op.cc:276] Timestamp: P: 1790408123613862 usec, L: 0
0926 07:35:33.657577 (+ 403us) write_op.cc:270] Start()
0926 07:35:33.657646 (+ 69us) write_op.cc:276] Timestamp: P: 1790408123654901 usec, L: 0
0926 07:35:33.658039 (+ 393us) write_op.cc:270] Start()
0926 07:35:33.658111 (+ 72us) write_op.cc:276] Timestamp: P: 1790408123665539 usec, L: 0
0926 07:35:33.658521 (+ 410us) write_op.cc:270] Start()
0926 07:35:33.658593 (+ 72us) write_op.cc:276] Timestamp: P: 1790408123699166 usec, L: 0
0926 07:35:33.704744 (+ 46151us) write_op.cc:270] Start()
0926 07:35:33.704857 (+ 113us) write_op.cc:276] Timestamp: P: 1790408123707909 usec, L: 0
0926 07:35:33.715521 (+ 10664us) write_op.cc:270] Start()
0926 07:35:33.715630 (+ 109us) write_op.cc:276] Timestamp: P: 1790408123800784 usec, L: 0
0926 07:35:33.716209 (+ 579us) write_op.cc:270] Start()
0926 07:35:33.716307 (+ 98us) write_op.cc:276] Timestamp: P: 1790408123927735 usec, L: 0
0926 07:35:33.716854 (+ 547us) write_op.cc:270] Start()
0926 07:35:33.716966 (+ 112us) write_op.cc:276] Timestamp: P: 1790408123946872 usec, L: 0
0926 07:35:33.717448 (+ 482us) write_op.cc:270] Start()
0926 07:35:33.717537 (+ 89us) write_op.cc:276] Timestamp: P: 1790408123963608 usec, L: 0
0926 07:35:33.717948 (+ 411us) write_op.cc:270] Start()
0926 07:35:33.718019 (+ 71us) write_op.cc:276] Timestamp: P: 1790408123999213 usec, L: 0
0926 07:35:33.718391 (+ 372us) write_op.cc:270] Start()
0926 07:35:33.718453 (+ 62us) write_op.cc:276] Timestamp: P: 1790408124007441 usec, L: 0
0926 07:35:33.718827 (+ 374us) write_op.cc:270] Start()
0926 07:35:33.718899 (+ 72us) write_op.cc:276] Timestamp: P: 1790408124025631 usec, L: 0
0926 07:35:33.719307 (+ 408us) write_op.cc:270] Start()
0926 07:35:33.719378 (+ 71us) write_op.cc:276] Timestamp: P: 1790408124094622 usec, L: 0
0926 07:35:33.719780 (+ 402us) write_op.cc:270] Start()
0926 07:35:33.719852 (+ 72us) write_op.cc:276] Timestamp: P: 1790408124120969 usec, L: 0
0926 07:35:33.720295 (+ 443us) write_op.cc:270] Start()
0926 07:35:33.720382 (+ 87us) write_op.cc:276] Timestamp: P: 1790408124170681 usec, L: 0
0926 07:35:33.720903 (+ 521us) write_op.cc:270] Start()
0926 07:35:33.721004 (+ 101us) write_op.cc:276] Timestamp: P: 1790408124192045 usec, L: 0
0926 07:35:33.721468 (+ 464us) write_op.cc:270] Start()
0926 07:35:33.721543 (+ 75us) write_op.cc:276] Timestamp: P: 1790408124234204 usec, L: 0
0926 07:35:33.722049 (+ 506us) write_op.cc:270] Start()
0926 07:35:33.722141 (+ 92us) write_op.cc:276] Timestamp: P: 1790408124244159 usec, L: 0
0926 07:35:33.722571 (+ 430us) write_op.cc:270] Start()
0926 07:35:33.722664 (+ 93us) write_op.cc:276] Timestamp: P: 1790408124273447 usec, L: 0
0926 07:35:33.736450 (+ 13786us) write_op.cc:270] Start()
0926 07:35:33.736537 (+ 87us) write_op.cc:276] Timestamp: P: 1790408124293987 usec, L: 0
0926 07:35:33.737016 (+ 479us) write_op.cc:270] Start()
0926 07:35:33.737103 (+ 87us) write_op.cc:276] Timestamp: P: 1790408124338182 usec, L: 0
0926 07:35:33.737531 (+ 428us) write_op.cc:270] Start()
0926 07:35:33.737606 (+ 75us) write_op.cc:276] Timestamp: P: 1790408124373326 usec, L: 0
0926 07:35:33.738033 (+ 427us) write_op.cc:270] Start()
0926 07:35:33.738112 (+ 79us) write_op.cc:276] Timestamp: P: 1790408124375056 usec, L: 0
0926 07:35:33.738545 (+ 433us) write_op.cc:270] Start()
0926 07:35:33.738620 (+ 75us) write_op.cc:276] Timestamp: P: 1790408124426056 usec, L: 0
0926 07:35:33.739059 (+ 439us) write_op.cc:270] Start()
0926 07:35:33.739134 (+ 75us) write_op.cc:276] Timestamp: P: 1790408124441331 usec, L: 0
0926 07:35:33.740940 (+ 1806us) write_op.cc:270] Start()
0926 07:35:33.741016 (+ 76us) write_op.cc:276] Timestamp: P: 1790408124572515 usec, L: 0
0926 07:35:33.741397 (+ 381us) write_op.cc:270] Start()
0926 07:35:33.741473 (+ 76us) write_op.cc:276] Timestamp: P: 1790408124575508 usec, L: 0
0926 07:35:33.746196 (+ 4723us) write_op.cc:270] Start()
0926 07:35:33.746295 (+ 99us) write_op.cc:276] Timestamp: P: 1790408124658223 usec, L: 0
0926 07:35:33.746806 (+ 511us) write_op.cc:270] Start()
0926 07:35:33.747068 (+ 262us) write_op.cc:276] Timestamp: P: 1790408124692973 usec, L: 0
0926 07:35:33.757762 (+ 10694us) write_op.cc:270] Start()
0926 07:35:33.757853 (+ 91us) write_op.cc:276] Timestamp: P: 1790408124796872 usec, L: 0
0926 07:35:33.758348 (+ 495us) write_op.cc:270] Start()
0926 07:35:33.758435 (+ 87us) write_op.cc:276] Timestamp: P: 1790408124846828 usec, L: 0
0926 07:35:33.758903 (+ 468us) write_op.cc:270] Start()
0926 07:35:33.758996 (+ 93us) write_op.cc:276] Timestamp: P: 1790408124936899 usec, L: 0
0926 07:35:33.759408 (+ 412us) write_op.cc:270] Start()
0926 07:35:33.759478 (+ 70us) write_op.cc:276] Timestamp: P: 1790408125005451 usec, L: 0
0926 07:35:33.759867 (+ 389us) write_op.cc:270] Start()
0926 07:35:33.759950 (+ 83us) write_op.cc:276] Timestamp: P: 1790408125013890 usec, L: 0
0926 07:35:33.760440 (+ 490us) write_op.cc:270] Start()
0926 07:35:33.760532 (+ 92us) write_op.cc:276] Timestamp: P: 1790408125017010 usec, L: 0
0926 07:35:33.761036 (+ 504us) write_op.cc:270] Start()
0926 07:35:33.761118 (+ 82us) write_op.cc:276] Timestamp: P: 1790408125141528 usec, L: 0
0926 07:35:33.761610 (+ 492us) write_op.cc:270] Start()
0926 07:35:33.761693 (+ 83us) write_op.cc:276] Timestamp: P: 1790408125202724 usec, L: 0
0926 07:35:33.762162 (+ 469us) write_op.cc:270] Start()
0926 07:35:33.762241 (+ 79us) write_op.cc:276] Timestamp: P: 1790408125206865 usec, L: 0
0926 07:35:33.762679 (+ 438us) write_op.cc:270] Start()
0926 07:35:33.762754 (+ 75us) write_op.cc:276] Timestamp: P: 1790408125283892 usec, L: 0
0926 07:35:33.763253 (+ 499us) write_op.cc:270] Start()
0926 07:35:33.763347 (+ 94us) write_op.cc:276] Timestamp: P: 1790408125317495 usec, L: 0
0926 07:35:33.763830 (+ 483us) write_op.cc:270] Start()
0926 07:35:33.763932 (+ 102us) write_op.cc:276] Timestamp: P: 1790408125345858 usec, L: 0
0926 07:35:33.772550 (+ 8618us) write_op.cc:270] Start()
0926 07:35:33.772626 (+ 76us) write_op.cc:276] Timestamp: P: 1790408125380127 usec, L: 0
0926 07:35:33.773099 (+ 473us) write_op.cc:270] Start()
0926 07:35:33.773180 (+ 81us) write_op.cc:276] Timestamp: P: 1790408125465224 usec, L: 0
0926 07:35:33.773668 (+ 488us) write_op.cc:270] Start()
0926 07:35:33.773761 (+ 93us) write_op.cc:276] Timestamp: P: 1790408125531884 usec, L: 0
0926 07:35:33.774271 (+ 510us) write_op.cc:270] Start()
0926 07:35:33.774350 (+ 79us) write_op.cc:276] Timestamp: P: 1790408125565376 usec, L: 0
0926 07:35:33.774800 (+ 450us) write_op.cc:270] Start()
0926 07:35:33.774893 (+ 93us) write_op.cc:276] Timestamp: P: 1790408125568594 usec, L: 0
0926 07:35:33.775343 (+ 450us) write_op.cc:270] Start()
0926 07:35:33.775427 (+ 84us) write_op.cc:276] Timestamp: P: 1790408125655752 usec, L: 0
0926 07:35:33.775860 (+ 433us) write_op.cc:270] Start()
0926 07:35:33.775949 (+ 89us) write_op.cc:276] Timestamp: P: 1790408125825164 usec, L: 0
0926 07:35:33.776911 (+ 962us) write_op.cc:270] Start()
0926 07:35:33.777004 (+ 93us) write_op.cc:276] Timestamp: P: 1790408125846217 usec, L: 0
0926 07:35:33.777513 (+ 509us) write_op.cc:270] Start()
0926 07:35:33.777602 (+ 89us) write_op.cc:276] Timestamp: P: 1790408125857124 usec, L: 0
0926 07:35:33.778120 (+ 518us) write_op.cc:270] Start()
0926 07:35:33.778213 (+ 93us) write_op.cc:276] Timestamp: P: 1790408125914472 usec, L: 0
0926 07:35:33.778693 (+ 480us) write_op.cc:270] Start()
0926 07:35:33.778782 (+ 89us) write_op.cc:276] Timestamp: P: 1790408125932955 usec, L: 0
0926 07:35:33.779237 (+ 455us) write_op.cc:270] Start()
0926 07:35:33.779305 (+ 68us) write_op.cc:276] Timestamp: P: 1790408125934645 usec, L: 0
0926 07:35:33.779706 (+ 401us) write_op.cc:270] Start()
0926 07:35:33.779787 (+ 81us) write_op.cc:276] Timestamp: P: 1790408125937499 usec, L: 0
0926 07:35:33.784371 (+ 4584us) write_op.cc:270] Start()
0926 07:35:33.784448 (+ 77us) write_op.cc:276] Timestamp: P: 1790408126122007 usec, L: 0
0926 07:35:33.784841 (+ 393us) write_op.cc:270] Start()
0926 07:35:33.784924 (+ 83us) write_op.cc:276] Timestamp: P: 1790408126142161 usec, L: 0
0926 07:35:33.785316 (+ 392us) write_op.cc:270] Start()
0926 07:35:33.785386 (+ 70us) write_op.cc:276] Timestamp: P: 1790408126153512 usec, L: 0
0926 07:35:33.785774 (+ 388us) write_op.cc:270] Start()
0926 07:35:33.785844 (+ 70us) write_op.cc:276] Timestamp: P: 1790408126257756 usec, L: 0
0926 07:35:33.786217 (+ 373us) write_op.cc:270] Start()
0926 07:35:33.786287 (+ 70us) write_op.cc:276] Timestamp: P: 1790408126273251 usec, L: 0
0926 07:35:33.786650 (+ 363us) write_op.cc:270] Start()
0926 07:35:33.786718 (+ 68us) write_op.cc:276] Timestamp: P: 1790408126274977 usec, L: 0
0926 07:35:33.787407 (+ 689us) write_op.cc:270] Start()
0926 07:35:33.787496 (+ 89us) write_op.cc:276] Timestamp: P: 1790408126277173 usec, L: 0
0926 07:35:33.787974 (+ 478us) write_op.cc:270] Start()
0926 07:35:33.788055 (+ 81us) write_op.cc:276] Timestamp: P: 1790408126280226 usec, L: 0
0926 07:35:33.792053 (+ 3998us) write_op.cc:270] Start()
0926 07:35:33.792292 (+ 239us) write_op.cc:276] Timestamp: P: 1790408126349435 usec, L: 0
0926 07:35:33.792817 (+ 525us) write_op.cc:270] Start()
0926 07:35:33.792914 (+ 97us) write_op.cc:276] Timestamp: P: 1790408126352287 usec, L: 0
0926 07:35:33.793448 (+ 534us) write_op.cc:270] Start()
0926 07:35:33.793548 (+ 100us) write_op.cc:276] Timestamp: P: 1790408126426942 usec, L: 0
0926 07:35:33.794085 (+ 537us) write_op.cc:270] Start()
0926 07:35:33.794184 (+ 99us) write_op.cc:276] Timestamp: P: 1790408126431138 usec, L: 0
0926 07:35:33.794725 (+ 541us) write_op.cc:270] Start()
0926 07:35:33.794806 (+ 81us) write_op.cc:276] Timestamp: P: 1790408126479854 usec, L: 0
0926 07:35:33.795295 (+ 489us) write_op.cc:270] Start()
0926 07:35:33.795381 (+ 86us) write_op.cc:276] Timestamp: P: 1790408127181478 usec, L: 0
0926 07:35:33.795822 (+ 441us) write_op.cc:270] Start()
0926 07:35:33.795915 (+ 93us) write_op.cc:276] Timestamp: P: 1790408127360456 usec, L: 0
0926 07:35:33.799541 (+ 3626us) write_op.cc:270] Start()
0926 07:35:33.799624 (+ 83us) write_op.cc:276] Timestamp: P: 1790408127364280 usec, L: 0
0926 07:35:33.800074 (+ 450us) write_op.cc:270] Start()
0926 07:35:33.800195 (+ 121us) write_op.cc:276] Timestamp: P: 1790408127499569 usec, L: 0
0926 07:35:33.804518 (+ 4323us) write_op.cc:270] Start()
0926 07:35:33.804590 (+ 72us) write_op.cc:276] Timestamp: P: 1790408127570664 usec, L: 0
0926 07:35:33.804985 (+ 395us) write_op.cc:270] Start()
0926 07:35:33.805046 (+ 61us) write_op.cc:276] Timestamp: P: 1790408127733832 usec, L: 0
0926 07:35:33.805458 (+ 412us) write_op.cc:270] Start()
0926 07:35:33.805531 (+ 73us) write_op.cc:276] Timestamp: P: 1790408127908316 usec, L: 0
0926 07:35:33.805917 (+ 386us) write_op.cc:270] Start()
0926 07:35:33.806000 (+ 83us) write_op.cc:276] Timestamp: P: 1790408127959152 usec, L: 0
0926 07:35:33.806472 (+ 472us) write_op.cc:270] Start()
0926 07:35:33.806553 (+ 81us) write_op.cc:276] Timestamp: P: 1790408127999547 usec, L: 0
0926 07:35:33.806951 (+ 398us) write_op.cc:270] Start()
0926 07:35:33.807010 (+ 59us) write_op.cc:276] Timestamp: P: 1790408128003539 usec, L: 0
0926 07:35:33.807419 (+ 409us) write_op.cc:270] Start()
0926 07:35:33.807505 (+ 86us) write_op.cc:276] Timestamp: P: 1790408128106565 usec, L: 0
0926 07:35:33.807940 (+ 435us) write_op.cc:270] Start()
0926 07:35:33.808001 (+ 61us) write_op.cc:276] Timestamp: P: 1790408128177232 usec, L: 0
0926 07:35:33.808413 (+ 412us) write_op.cc:270] Start()
0926 07:35:33.808474 (+ 61us) write_op.cc:276] Timestamp: P: 1790408128179279 usec, L: 0
0926 07:35:33.808840 (+ 366us) write_op.cc:270] Start()
0926 07:35:33.808917 (+ 77us) write_op.cc:276] Timestamp: P: 1790408128302268 usec, L: 0
0926 07:35:33.809286 (+ 369us) write_op.cc:270] Start()
0926 07:35:33.809355 (+ 69us) write_op.cc:276] Timestamp: P: 1790408128306964 usec, L: 0
0926 07:35:33.836689 (+ 27334us) write_op.cc:270] Start()
0926 07:35:33.836786 (+ 97us) write_op.cc:276] Timestamp: P: 1790408128337598 usec, L: 0
0926 07:35:33.837260 (+ 474us) write_op.cc:270] Start()
0926 07:35:33.837335 (+ 75us) write_op.cc:276] Timestamp: P: 1790408128339681 usec, L: 0
0926 07:35:33.837725 (+ 390us) write_op.cc:270] Start()
0926 07:35:33.837798 (+ 73us) write_op.cc:276] Timestamp: P: 1790408128347441 usec, L: 0
0926 07:35:33.838211 (+ 413us) write_op.cc:270] Start()
0926 07:35:33.838281 (+ 70us) write_op.cc:276] Timestamp: P: 1790408128420353 usec, L: 0
0926 07:35:33.838665 (+ 384us) write_op.cc:270] Start()
0926 07:35:33.838734 (+ 69us) write_op.cc:276] Timestamp: P: 1790408128467884 usec, L: 0
0926 07:35:33.839152 (+ 418us) write_op.cc:270] Start()
0926 07:35:33.839222 (+ 70us) write_op.cc:276] Timestamp: P: 1790408128477093 usec, L: 0
0926 07:35:33.839585 (+ 363us) write_op.cc:270] Start()
0926 07:35:33.839654 (+ 69us) write_op.cc:276] Timestamp: P: 1790408128565347 usec, L: 0
0926 07:35:33.840082 (+ 428us) write_op.cc:270] Start()
0926 07:35:33.840202 (+ 120us) write_op.cc:276] Timestamp: P: 1790408128568506 usec, L: 0
0926 07:35:33.840624 (+ 422us) write_op.cc:270] Start()
0926 07:35:33.840690 (+ 66us) write_op.cc:276] Timestamp: P: 1790408128690123 usec, L: 0
0926 07:35:33.841087 (+ 397us) write_op.cc:270] Start()
0926 07:35:33.841153 (+ 66us) write_op.cc:276] Timestamp: P: 1790408128762209 usec, L: 0
0926 07:35:33.841555 (+ 402us) write_op.cc:270] Start()
0926 07:35:33.841626 (+ 71us) write_op.cc:276] Timestamp: P: 1790408128858426 usec, L: 0
0926 07:35:33.842006 (+ 380us) write_op.cc:270] Start()
0926 07:35:33.842068 (+ 62us) write_op.cc:276] Timestamp: P: 1790408128861648 usec, L: 0
0926 07:35:33.842423 (+ 355us) write_op.cc:270] Start()
0926 07:35:33.842493 (+ 70us) write_op.cc:276] Timestamp: P: 1790408128916386 usec, L: 0
0926 07:35:33.842910 (+ 417us) write_op.cc:270] Start()
0926 07:35:33.842971 (+ 61us) write_op.cc:276] Timestamp: P: 1790408128943190 usec, L: 0
0926 07:35:33.843339 (+ 368us) write_op.cc:270] Start()
0926 07:35:33.843407 (+ 68us) write_op.cc:276] Timestamp: P: 1790408129019931 usec, L: 0
0926 07:35:33.843769 (+ 362us) write_op.cc:270] Start()
0926 07:35:33.843833 (+ 64us) write_op.cc:276] Timestamp: P: 1790408129030735 usec, L: 0
0926 07:35:33.844341 (+ 508us) write_op.cc:270] Start()
0926 07:35:33.844408 (+ 67us) write_op.cc:276] Timestamp: P: 1790408129088397 usec, L: 0
0926 07:35:33.844780 (+ 372us) write_op.cc:270] Start()
0926 07:35:33.844844 (+ 64us) write_op.cc:276] Timestamp: P: 1790408129213896 usec, L: 0
0926 07:35:33.845754 (+ 910us) write_op.cc:270] Start()
0926 07:35:33.845846 (+ 92us) write_op.cc:276] Timestamp: P: 1790408129233156 usec, L: 0
0926 07:35:33.846354 (+ 508us) write_op.cc:270] Start()
0926 07:35:33.846432 (+ 78us) write_op.cc:276] Timestamp: P: 1790408129288397 usec, L: 0
0926 07:35:33.846845 (+ 413us) write_op.cc:270] Start()
0926 07:35:33.846914 (+ 69us) write_op.cc:276] Timestamp: P: 1790408129346568 usec, L: 0
0926 07:35:33.847261 (+ 347us) write_op.cc:270] Start()
0926 07:35:33.847336 (+ 75us) write_op.cc:276] Timestamp: P: 1790408129366091 usec, L: 0
0926 07:35:33.860855 (+ 13519us) write_op.cc:270] Start()
0926 07:35:33.860962 (+ 107us) write_op.cc:276] Timestamp: P: 1790408129427053 usec, L: 0
0926 07:35:33.861443 (+ 481us) write_op.cc:270] Start()
0926 07:35:33.861510 (+ 67us) write_op.cc:276] Timestamp: P: 1790408129452574 usec, L: 0
0926 07:35:33.861937 (+ 427us) write_op.cc:270] Start()
0926 07:35:33.862006 (+ 69us) write_op.cc:276] Timestamp: P: 1790408129575787 usec, L: 0
0926 07:35:33.868412 (+ 6406us) write_op.cc:270] Start()
0926 07:35:33.868491 (+ 79us) write_op.cc:276] Timestamp: P: 1790408129654110 usec, L: 0
0926 07:35:33.869010 (+ 519us) write_op.cc:270] Start()
0926 07:35:33.869101 (+ 91us) write_op.cc:276] Timestamp: P: 1790408129690895 usec, L: 0
0926 07:35:33.869542 (+ 441us) write_op.cc:270] Start()
0926 07:35:33.869621 (+ 79us) write_op.cc:276] Timestamp: P: 1790408129794945 usec, L: 0
0926 07:35:33.880831 (+ 11210us) write_op.cc:270] Start()
0926 07:35:33.880922 (+ 91us) write_op.cc:276] Timestamp: P: 1790408129852909 usec, L: 0
0926 07:35:33.881286 (+ 364us) write_op.cc:270] Start()
0926 07:35:33.881353 (+ 67us) write_op.cc:276] Timestamp: P: 1790408129954029 usec, L: 0
0926 07:35:33.881851 (+ 498us) write_op.cc:270] Start()
0926 07:35:33.881947 (+ 96us) write_op.cc:276] Timestamp: P: 1790408129965571 usec, L: 0
0926 07:35:33.882376 (+ 429us) write_op.cc:270] Start()
0926 07:35:33.882436 (+ 60us) write_op.cc:276] Timestamp: P: 1790408130145881 usec, L: 0
0926 07:35:33.882804 (+ 368us) write_op.cc:270] Start()
0926 07:35:33.882862 (+ 58us) write_op.cc:276] Timestamp: P: 1790408130256186 usec, L: 0
0926 07:35:33.883204 (+ 342us) write_op.cc:270] Start()
0926 07:35:33.883265 (+ 61us) write_op.cc:276] Timestamp: P: 1790408130333737 usec, L: 0
0926 07:35:33.883619 (+ 354us) write_op.cc:270] Start()
0926 07:35:33.883678 (+ 59us) write_op.cc:276] Timestamp: P: 1790408130398327 usec, L: 0
0926 07:35:33.884032 (+ 354us) write_op.cc:270] Start()
0926 07:35:33.884089 (+ 57us) write_op.cc:276] Timestamp: P: 1790408130462969 usec, L: 0
0926 07:35:33.885125 (+ 1036us) write_op.cc:270] Start()
0926 07:35:33.885192 (+ 67us) write_op.cc:276] Timestamp: P: 1790408130475292 usec, L: 0
0926 07:35:33.885519 (+ 327us) write_op.cc:270] Start()
0926 07:35:33.885573 (+ 54us) write_op.cc:276] Timestamp: P: 1790408130486248 usec, L: 0
0926 07:35:33.887157 (+ 1584us) write_op.cc:270] Start()
0926 07:35:33.887237 (+ 80us) write_op.cc:276] Timestamp: P: 1790408130487956 usec, L: 0
0926 07:35:33.887633 (+ 396us) write_op.cc:270] Start()
0926 07:35:33.887707 (+ 74us) write_op.cc:276] Timestamp: P: 1790408130651593 usec, L: 0
0926 07:35:33.888200 (+ 493us) write_op.cc:270] Start()
0926 07:35:33.888290 (+ 90us) write_op.cc:276] Timestamp: P: 1790408130664378 usec, L: 0
0926 07:35:33.888669 (+ 379us) write_op.cc:270] Start()
0926 07:35:33.888733 (+ 64us) write_op.cc:276] Timestamp: P: 1790408130677819 usec, L: 0
0926 07:35:33.889169 (+ 436us) write_op.cc:270] Start()
0926 07:35:33.889232 (+ 63us) write_op.cc:276] Timestamp: P: 1790408130710703 usec, L: 0
0926 07:35:33.889608 (+ 376us) write_op.cc:270] Start()
0926 07:35:33.889684 (+ 76us) write_op.cc:276] Timestamp: P: 1790408130843656 usec, L: 0
0926 07:35:33.890123 (+ 439us) write_op.cc:270] Start()
0926 07:35:33.890216 (+ 93us) write_op.cc:276] Timestamp: P: 1790408130938050 usec, L: 0
0926 07:35:33.890585 (+ 369us) write_op.cc:270] Start()
0926 07:35:33.890652 (+ 67us) write_op.cc:276] Timestamp: P: 1790408131011171 usec, L: 0
0926 07:35:33.891070 (+ 418us) write_op.cc:270] Start()
0926 07:35:33.891140 (+ 70us) write_op.cc:276] Timestamp: P: 1790408
I20260926 07:35:35.569957 12291 raft_consensus.cc:482] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:35.570238 12291 raft_consensus.cc:3037] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:35.597168 11118 master_service.cc:946] ReplaceTablet: received request to replace tablet 0b900e55dcb34eef9f76de31b389e12d from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:35.601598 12291 raft_consensus.cc:504] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.603936 11754 tablet_bootstrap.cc:492] T b8e926ee35854c28a8c698384526fe38 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:35.604416 11754 ts_tablet_manager.cc:1397] T b8e926ee35854c28a8c698384526fe38 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.139s user 0.007s sys 0.001s
I20260926 07:35:35.604873 12291 leader_election.cc:288] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:35.604830 11604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cd482fb159d418291cb02bc9138b076" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:35.605471 11604 raft_consensus.cc:3037] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:35.606757 11754 raft_consensus.cc:348] T b8e926ee35854c28a8c698384526fe38 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.607508 11754 raft_consensus.cc:374] T b8e926ee35854c28a8c698384526fe38 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:35.607790 11754 raft_consensus.cc:729] T b8e926ee35854c28a8c698384526fe38 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:35.608474 11754 consensus_queue.cc:260] T b8e926ee35854c28a8c698384526fe38 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.613420 12296 raft_consensus.cc:482] T ed9f7faf55684e3e8512a033aa1443fe P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:35.612957 12203 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "b6e2fc900b374ebab759831b571a208d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } } }
I20260926 07:35:35.613760 12203 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d [sys.catalog]: This master's current role is: LEADER
I20260926 07:35:35.613801 12296 raft_consensus.cc:504] T ed9f7faf55684e3e8512a033aa1443fe P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.615540 12296 leader_election.cc:288] T ed9f7faf55684e3e8512a033aa1443fe P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:35.617386 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9f7faf55684e3e8512a033aa1443fe" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
W20260926 07:35:35.618474 11545 leader_election.cc:341] T ed9f7faf55684e3e8512a033aa1443fe P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:35.642499 11746 tablet_bootstrap.cc:654] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:35.648674 12292 raft_consensus.cc:504] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:35.654807 11754 ts_tablet_manager.cc:1428] T b8e926ee35854c28a8c698384526fe38 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.050s user 0.004s sys 0.001s
I20260926 07:35:35.655777 11754 tablet_bootstrap.cc:492] T 52d064b684d0491e91f176d24eddabe1 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:35.665026 11604 raft_consensus.cc:2454] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:35.665985 11402 leader_election.cc:302] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:35.666671 12293 raft_consensus.cc:2781] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:35.667085 12293 raft_consensus.cc:686] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:35.667811 12293 consensus_queue.cc:237] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.703465 11607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248ab84a1f284c2bbddf62f00b485e13" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:35.704107 11607 raft_consensus.cc:3037] T 248ab84a1f284c2bbddf62f00b485e13 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:35.712066 11746 tablet_bootstrap.cc:492] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:35.712549 11746 ts_tablet_manager.cc:1397] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.172s user 0.004s sys 0.008s
I20260926 07:35:35.716362 12346 tablet.cc:1722] T 0218d2a727f942e48d770878e6bf4a54 P 22f3902d01584936b20437606cb86dd1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260926 07:35:35.719030 12292 leader_election.cc:288] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:35.718587 12345 tablet.cc:1722] T 0218d2a727f942e48d770878e6bf4a54 P 6070e9e3d1234a9d83e49431416afade: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260926 07:35:35.760408 11754 tablet_bootstrap.cc:654] T 52d064b684d0491e91f176d24eddabe1 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:35.743548 11607 raft_consensus.cc:2454] T 248ab84a1f284c2bbddf62f00b485e13 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:35.774676 11746 raft_consensus.cc:348] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.783258 11756 raft_consensus.cc:348] T 7e52ec7d802a4e90bccef777f77a2a1b P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:35.783939 11756 raft_consensus.cc:374] T 7e52ec7d802a4e90bccef777f77a2a1b P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:35.784296 11756 raft_consensus.cc:729] T 7e52ec7d802a4e90bccef777f77a2a1b P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:35.785049 11756 consensus_queue.cc:260] T 7e52ec7d802a4e90bccef777f77a2a1b P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:35.786942 11313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cd482fb159d418291cb02bc9138b076" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:35.787598 11314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248ab84a1f284c2bbddf62f00b485e13" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
W20260926 07:35:35.788326 11401 leader_election.cc:341] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
W20260926 07:35:35.789575 11401 leader_election.cc:341] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:35.792397 11316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9f7faf55684e3e8512a033aa1443fe" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:35.793558 11311 raft_consensus.cc:3037] T 90f62593b6f844fc8a41b9be240101fd P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:35.796586 12343 tablet.cc:1722] T 0218d2a727f942e48d770878e6bf4a54 P e9bfebb34fa14cd2ad4843fffbe5c54e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260926 07:35:35.775375 11746 raft_consensus.cc:374] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:35.804270 11746 raft_consensus.cc:729] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:35.805143 11746 consensus_queue.cc:260] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.794111 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cd482fb159d418291cb02bc9138b076" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
W20260926 07:35:35.809909 11401 leader_election.cc:341] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:35.811731 11746 ts_tablet_manager.cc:1428] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.099s user 0.004s sys 0.001s
I20260926 07:35:35.829316 12168 ts_tablet_manager.cc:1918] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:35.829777 12168 log.cc:1196] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/a8b629a4aba045b697e33bbebecb05b7
I20260926 07:35:35.830555 12168 ts_tablet_manager.cc:1939] T a8b629a4aba045b697e33bbebecb05b7 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
W20260926 07:35:35.810387 11546 leader_election.cc:341] T ed9f7faf55684e3e8512a033aa1443fe P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:35.833361 11546 leader_election.cc:302] T ed9f7faf55684e3e8512a033aa1443fe P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:35.835219 12296 raft_consensus.cc:2726] T ed9f7faf55684e3e8512a033aa1443fe P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:35.842475 12296 raft_consensus.cc:482] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 22f3902d01584936b20437606cb86dd1)
I20260926 07:35:35.842945 12296 raft_consensus.cc:504] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.844195 11746 tablet_bootstrap.cc:492] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:35.786800 11756 ts_tablet_manager.cc:1428] T 7e52ec7d802a4e90bccef777f77a2a1b P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.350s user 0.011s sys 0.000s
I20260926 07:35:35.844811 12296 leader_election.cc:288] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:35.845253 11756 tablet_bootstrap.cc:492] T 04180ace5d4548dfbcd226bf229d5c26 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
W20260926 07:35:35.847378 11545 leader_election.cc:341] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Not found: Tablet not found: 55ad1d96e2314d1eb0a717a3e9ff31bc
W20260926 07:35:35.850656 11546 leader_election.cc:341] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Not found: Tablet not found: 55ad1d96e2314d1eb0a717a3e9ff31bc
I20260926 07:35:35.850996 11546 leader_election.cc:302] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:35.852757 12296 raft_consensus.cc:2726] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:35:35.821728 11871 ts_tablet_manager.cc:1918] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:35.853356 11871 log.cc:1196] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/3e28ca159d8349748812825ac1947c51
I20260926 07:35:35.854058 11871 ts_tablet_manager.cc:1939] T 3e28ca159d8349748812825ac1947c51 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:35.858916 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 131) took 5086 ms. Trace:
I20260926 07:35:35.860684 11871 rpcz_store.cc:276] 0926 07:35:30.772870 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:30.773173 (+ 303us) service_pool.cc:227] Handling call
0926 07:35:35.381514 (+4608341us) ts_tablet_manager.cc:1145] Deleting tablet 3e28ca159d8349748812825ac1947c51
0926 07:35:35.381558 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:35.514317 (+132759us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:35.821520 (+307203us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:35.858862 (+ 37342us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":61361,"tablet-delete.queue_time_us":4607846}
I20260926 07:35:35.867652 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 3e28ca159d8349748812825ac1947c51 (table default.Table_fd842e14c4984402b2b1b77baa1f9360 [id=cc4663f9d1734d9cb4cc4e1f061310a3]) successfully deleted
I20260926 07:35:35.868737 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 133) took 5052 ms. Trace:
I20260926 07:35:35.872155 11402 leader_election.cc:302] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters: e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:35.872670 11871 rpcz_store.cc:276] 0926 07:35:30.816800 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:30.817418 (+ 618us) service_pool.cc:227] Handling call
0926 07:35:35.868438 (+5051020us) ts_tablet_manager.cc:1145] Deleting tablet 21919d22b1d24546911d36384b01d9cc
0926 07:35:35.868475 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:35.868690 (+ 215us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5050518}
I20260926 07:35:35.873078 12293 raft_consensus.cc:2781] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:35.873972 12354 raft_consensus.cc:482] T f8f025d645eb478f95fa0aef17a86729 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260926 07:35:35.874560 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 21919d22b1d24546911d36384b01d9cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 21919d22b1d24546911d36384b01d9cc already in progress: creating tablet
I20260926 07:35:35.874356 12354 raft_consensus.cc:504] T f8f025d645eb478f95fa0aef17a86729 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.870002 11754 tablet_bootstrap.cc:492] T 52d064b684d0491e91f176d24eddabe1 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:35.875372 11754 ts_tablet_manager.cc:1397] T 52d064b684d0491e91f176d24eddabe1 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.220s user 0.007s sys 0.005s
I20260926 07:35:35.877918 11754 raft_consensus.cc:348] T 52d064b684d0491e91f176d24eddabe1 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.881799 11756 tablet_bootstrap.cc:654] T 04180ace5d4548dfbcd226bf229d5c26 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:35.882092 11754 raft_consensus.cc:374] T 52d064b684d0491e91f176d24eddabe1 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:35.882383 11754 raft_consensus.cc:729] T 52d064b684d0491e91f176d24eddabe1 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:35.873482 12293 raft_consensus.cc:686] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:35.883585 12293 consensus_queue.cc:237] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:35.887456 11746 tablet_bootstrap.cc:654] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:35.867061 12259 ts_tablet_manager.cc:1918] T a8b629a4aba045b697e33bbebecb05b7 P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:35.889580 12259 log.cc:1196] T a8b629a4aba045b697e33bbebecb05b7 P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/a8b629a4aba045b697e33bbebecb05b7
I20260926 07:35:35.890627 12259 ts_tablet_manager.cc:1939] T a8b629a4aba045b697e33bbebecb05b7 P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:35.893006 12259 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 160) took 1859 ms. Trace:
I20260926 07:35:35.878830 11871 tablet_replica.cc:330] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:35.894176 11871 raft_consensus.cc:2229] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:35.898147 11871 raft_consensus.cc:2258] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:35.900883 11871 ts_tablet_manager.cc:1905] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:35.900218 12259 rpcz_store.cc:276] 0926 07:35:34.033142 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.033228 (+ 86us) service_pool.cc:227] Handling call
0926 07:35:35.382869 (+1349641us) ts_tablet_manager.cc:1145] Deleting tablet a8b629a4aba045b697e33bbebecb05b7
0926 07:35:35.382905 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:35.559637 (+176732us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:35.866822 (+307185us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:35.892958 (+ 26136us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1349202}
I20260926 07:35:35.901703 12259 tablet_replica.cc:330] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:35.901671 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet a8b629a4aba045b697e33bbebecb05b7 (table default.Table_de0d2fdff98146f9aede70ab6a51dbec [id=c469a15ec981495a879b63ec5f961277]) successfully deleted
I20260926 07:35:35.903229 12259 raft_consensus.cc:2229] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:35.906294 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8f025d645eb478f95fa0aef17a86729" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:35.906776 11311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8f025d645eb478f95fa0aef17a86729" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:35.897709 11754 consensus_queue.cc:260] T 52d064b684d0491e91f176d24eddabe1 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
W20260926 07:35:35.907502 11545 leader_election.cc:341] T f8f025d645eb478f95fa0aef17a86729 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
W20260926 07:35:35.908484 11546 leader_election.cc:341] T f8f025d645eb478f95fa0aef17a86729 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:35.908825 11546 leader_election.cc:302] T f8f025d645eb478f95fa0aef17a86729 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:35.909543 12296 raft_consensus.cc:2726] T f8f025d645eb478f95fa0aef17a86729 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:35.911617 11754 ts_tablet_manager.cc:1428] T 52d064b684d0491e91f176d24eddabe1 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.036s user 0.001s sys 0.004s
I20260926 07:35:35.917469 12168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 137) took 3926 ms. Trace:
I20260926 07:35:35.917865 12168 rpcz_store.cc:276] 0926 07:35:31.991491 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:32.031839 (+ 40348us) service_pool.cc:227] Handling call
0926 07:35:35.260544 (+3228705us) ts_tablet_manager.cc:1145] Deleting tablet a8b629a4aba045b697e33bbebecb05b7
0926 07:35:35.260649 (+ 105us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:35.526793 (+266144us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:35.829080 (+302287us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:35.917407 (+ 88327us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3148118}
I20260926 07:35:35.919503 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet a8b629a4aba045b697e33bbebecb05b7 (table default.Table_de0d2fdff98146f9aede70ab6a51dbec [id=c469a15ec981495a879b63ec5f961277]) successfully deleted
I20260926 07:35:35.920022 12168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 147) took 3183 ms. Trace:
I20260926 07:35:35.920393 12168 rpcz_store.cc:276] 0926 07:35:32.736922 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:32.737049 (+ 127us) service_pool.cc:227] Handling call
0926 07:35:35.919642 (+3182593us) ts_tablet_manager.cc:1145] Deleting tablet 6d524be8509645a89a75b4bac15f0889
0926 07:35:35.919693 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:35.919974 (+ 281us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3182056}
W20260926 07:35:35.921383 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 6d524be8509645a89a75b4bac15f0889 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d524be8509645a89a75b4bac15f0889 already in progress: creating tablet
I20260926 07:35:35.921638 12168 tablet_replica.cc:330] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:35.922430 12168 raft_consensus.cc:2229] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:35.923013 12168 raft_consensus.cc:2258] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:35.923522 12354 leader_election.cc:288] T f8f025d645eb478f95fa0aef17a86729 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:35.936358 11754 tablet_bootstrap.cc:492] T 25b8b701563344bbbc986db2b0f39b9b P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:35.936929 11746 tablet_bootstrap.cc:492] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:35.937362 11746 ts_tablet_manager.cc:1397] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.093s user 0.004s sys 0.009s
I20260926 07:35:35.940093 11746 raft_consensus.cc:348] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.944347 12259 raft_consensus.cc:2258] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:35.945137 12293 raft_consensus.cc:482] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:35.947794 11746 raft_consensus.cc:374] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260926 07:35:35.948805 11546 consensus_peers.cc:487] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:35.948942 11746 raft_consensus.cc:729] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:35.948581 12293 raft_consensus.cc:504] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.951043 12293 leader_election.cc:288] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:35.951465 11606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef2e44be4084fc78e61cf822659b4ea" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:35.952065 11606 raft_consensus.cc:2454] T cef2e44be4084fc78e61cf822659b4ea P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:35.953290 11402 leader_election.cc:302] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:35.953891 12293 raft_consensus.cc:2781] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:35.954238 12293 raft_consensus.cc:482] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:35.954579 12293 raft_consensus.cc:3037] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:35.954602 11311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef2e44be4084fc78e61cf822659b4ea" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
W20260926 07:35:35.955760 11401 leader_election.cc:341] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:35.964736 12168 ts_tablet_manager.cc:1905] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:35.964370 11746 consensus_queue.cc:260] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.974866 12259 ts_tablet_manager.cc:1905] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:35.979789 11746 ts_tablet_manager.cc:1428] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.042s user 0.000s sys 0.006s
I20260926 07:35:35.980800 11756 tablet_bootstrap.cc:492] T 04180ace5d4548dfbcd226bf229d5c26 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:35.981328 11756 ts_tablet_manager.cc:1397] T 04180ace5d4548dfbcd226bf229d5c26 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.136s user 0.009s sys 0.003s
I20260926 07:35:35.985406 12293 raft_consensus.cc:504] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.987835 11606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef2e44be4084fc78e61cf822659b4ea" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:35.988409 11606 raft_consensus.cc:3037] T cef2e44be4084fc78e61cf822659b4ea P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:35.995564 11606 raft_consensus.cc:2454] T cef2e44be4084fc78e61cf822659b4ea P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:35.996596 11402 leader_election.cc:302] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:35.997277 12292 raft_consensus.cc:2781] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:35.997732 11746 tablet_bootstrap.cc:492] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:35.997699 12292 raft_consensus.cc:686] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:35.998845 12292 consensus_queue.cc:237] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:35.984050 11756 raft_consensus.cc:348] T 04180ace5d4548dfbcd226bf229d5c26 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:36.007718 11756 raft_consensus.cc:374] T 04180ace5d4548dfbcd226bf229d5c26 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:36.008342 11756 raft_consensus.cc:729] T 04180ace5d4548dfbcd226bf229d5c26 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:36.020272 11754 tablet_bootstrap.cc:654] T 25b8b701563344bbbc986db2b0f39b9b P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:36.025146 11871 ts_tablet_manager.cc:1918] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:36.018086 11756 consensus_queue.cc:260] T 04180ace5d4548dfbcd226bf229d5c26 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:36.030591 11311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef2e44be4084fc78e61cf822659b4ea" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
W20260926 07:35:36.031836 11401 leader_election.cc:341] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:36.033859 11756 ts_tablet_manager.cc:1428] T 04180ace5d4548dfbcd226bf229d5c26 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.052s user 0.003s sys 0.002s
I20260926 07:35:36.037629 12293 leader_election.cc:288] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:36.041688 11871 log.cc:1196] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/04180ace5d4548dfbcd226bf229d5c26
I20260926 07:35:36.044951 12344 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b6e2fc900b374ebab759831b571a208d)
I20260926 07:35:36.045351 11871 ts_tablet_manager.cc:1939] T 04180ace5d4548dfbcd226bf229d5c26 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:36.045426 12344 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:36.040206 11746 tablet_bootstrap.cc:654] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:36.053061 11756 tablet_bootstrap.cc:492] T 248ab84a1f284c2bbddf62f00b485e13 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:36.070290 12029 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" instance_seqno: 1790408114853442) as {username='slave'} at 127.10.171.1:42417; Asking this server to re-register.
I20260926 07:35:36.047883 12344 leader_election.cc:288] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0773380643194748af6701300513e7a9 (127.10.171.61:38823), b6e2fc900b374ebab759831b571a208d (127.10.171.60:40073)
I20260926 07:35:36.047231 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 134) took 5139 ms. Trace:
I20260926 07:35:36.073559 11130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6ef5b900d0f4e73a358f415efa97249" candidate_term: 4 candidate_status { last_received { term: 3 index: 269 } } ignore_live_leader: false dest_uuid: "b6e2fc900b374ebab759831b571a208d" is_pre_election: true
I20260926 07:35:36.075711 11370 heartbeater.cc:464] Registering TS with master...
I20260926 07:35:36.076301 12304 raft_consensus.cc:482] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:36.076787 12304 raft_consensus.cc:504] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:36.078930 12304 leader_election.cc:288] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:36.080865 11606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52d064b684d0491e91f176d24eddabe1" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:36.081599 11606 raft_consensus.cc:2454] T 52d064b684d0491e91f176d24eddabe1 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:36.083005 11311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52d064b684d0491e91f176d24eddabe1" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:36.083325 11402 leader_election.cc:302] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
W20260926 07:35:36.084432 11401 leader_election.cc:341] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:36.085386 12304 raft_consensus.cc:2781] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:36.085780 12304 raft_consensus.cc:482] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:36.086283 12304 raft_consensus.cc:3037] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:36.072414 11871 rpcz_store.cc:276] 0926 07:35:30.907632 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:30.907758 (+ 126us) service_pool.cc:227] Handling call
0926 07:35:35.878695 (+4970937us) ts_tablet_manager.cc:1145] Deleting tablet 04180ace5d4548dfbcd226bf229d5c26
0926 07:35:35.878740 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:35.958297 (+ 79557us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:36.024951 (+ 66654us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:36.047173 (+ 22222us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4969909}
I20260926 07:35:36.095290 11754 tablet_bootstrap.cc:492] T 25b8b701563344bbbc986db2b0f39b9b P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:36.095031 11296 tablet_service.cc:1467] Processing CreateTablet for tablet eb9878f5b28f4d6589d838f70af464c8 (DEFAULT_TABLE table=default.Table_d6bf3b3f4b934ec093561293afadaea1 [id=d14935a6188c4e5387f0c79838d4f708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.095690 11754 ts_tablet_manager.cc:1397] T 25b8b701563344bbbc986db2b0f39b9b P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.159s user 0.006s sys 0.006s
I20260926 07:35:36.096354 11296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb9878f5b28f4d6589d838f70af464c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.097858 11754 raft_consensus.cc:348] T 25b8b701563344bbbc986db2b0f39b9b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:36.098450 11754 raft_consensus.cc:374] T 25b8b701563344bbbc986db2b0f39b9b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:36.098690 11754 raft_consensus.cc:729] T 25b8b701563344bbbc986db2b0f39b9b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
W20260926 07:35:36.099258 11402 consensus_peers.cc:487] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 -> Peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Couldn't send request to peer 6070e9e3d1234a9d83e49431416afade. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 04180ace5d4548dfbcd226bf229d5c26. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:36.095397 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 136) took 4473 ms. Trace:
I20260926 07:35:36.100363 11871 rpcz_store.cc:276] 0926 07:35:31.622270 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:31.624422 (+ 2152us) service_pool.cc:227] Handling call
0926 07:35:36.095102 (+4470680us) ts_tablet_manager.cc:1145] Deleting tablet 30df9b4afb414c90af5f4fb0c4582f01
0926 07:35:36.095138 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:36.095352 (+ 214us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4470054}
W20260926 07:35:36.103645 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 30df9b4afb414c90af5f4fb0c4582f01 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 30df9b4afb414c90af5f4fb0c4582f01 already in progress: creating tablet
I20260926 07:35:36.104626 11871 tablet_replica.cc:330] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:36.106038 11871 raft_consensus.cc:2229] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:36.097334 12304 raft_consensus.cc:504] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:36.109308 11118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 0b900e55dcb34eef9f76de31b389e12d
I20260926 07:35:36.109658 11113 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 33ca5d33a5184ce3b4fe88a9a6755124
I20260926 07:35:36.110455 11118 catalog_manager.cc:6565] ReplaceTablet: tablet 0b900e55dcb34eef9f76de31b389e12d deleted and replaced by tablet 3e1ea29b71c64497bb427c575b78fe01
I20260926 07:35:36.111377 11113 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_da7cb8d741f244f797eb0847d8500737" } new_table_name: "default.Table_1330df49c5964507ba4c568038f33499" modify_external_catalogs: true
I20260926 07:35:36.111250 11118 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_a4c0b35e65b84427bc8f867d5717dc3a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:36.113273 11285 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b900e55dcb34eef9f76de31b389e12d with delete_type TABLET_DATA_DELETED (replaced by tablet 3e1ea29b71c64497bb427c575b78fe01) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:36.113919 11289 tablet_service.cc:1514] Processing DeleteTablet for tablet 33ca5d33a5184ce3b4fe88a9a6755124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:35 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:36.112012 11606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52d064b684d0491e91f176d24eddabe1" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:36.114369 11606 raft_consensus.cc:3037] T 52d064b684d0491e91f176d24eddabe1 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:36.114598 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b900e55dcb34eef9f76de31b389e12d with delete_type TABLET_DATA_DELETED (replaced by tablet 3e1ea29b71c64497bb427c575b78fe01) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:36.115149 11437 tablet_service.cc:1514] Processing DeleteTablet for tablet 33ca5d33a5184ce3b4fe88a9a6755124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:35 UTC) from {username='slave'} at 127.0.0.1:40244
W20260926 07:35:36.116133 11118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a4c0b35e65b84427bc8f867d5717dc3a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:36.113273 11581 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b900e55dcb34eef9f76de31b389e12d with delete_type TABLET_DATA_DELETED (replaced by tablet 3e1ea29b71c64497bb427c575b78fe01) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:36.113399 11580 tablet_service.cc:1514] Processing DeleteTablet for tablet 33ca5d33a5184ce3b4fe88a9a6755124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:35 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:36.119542 11746 tablet_bootstrap.cc:492] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:36.119901 11746 ts_tablet_manager.cc:1397] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.123s user 0.001s sys 0.009s
I20260926 07:35:36.121968 11746 raft_consensus.cc:348] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:36.122536 11746 raft_consensus.cc:374] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:36.122792 11746 raft_consensus.cc:729] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
W20260926 07:35:36.127753 11986 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.60:40073, user_credentials={real_user=slave}} blocked reactor thread for 46708.7us
I20260926 07:35:36.132748 11051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6ef5b900d0f4e73a358f415efa97249" candidate_term: 4 candidate_status { last_received { term: 3 index: 269 } } ignore_live_leader: false dest_uuid: "0773380643194748af6701300513e7a9" is_pre_election: true
I20260926 07:35:36.134539 11981 leader_election.cc:302] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6ef5b900d0f4e73a358f415efa97249; no voters: 0773380643194748af6701300513e7a9, b6e2fc900b374ebab759831b571a208d
I20260926 07:35:36.136559 12344 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d6ef5b900d0f4e73a358f415efa97249 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260926 07:35:36.146559 11606 raft_consensus.cc:2454] T 52d064b684d0491e91f176d24eddabe1 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:36.150413 11402 leader_election.cc:302] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:36.151182 12293 raft_consensus.cc:2781] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:36.151825 12293 raft_consensus.cc:686] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:36.148051 11754 consensus_queue.cc:260] T 25b8b701563344bbbc986db2b0f39b9b P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:36.153373 12293 consensus_queue.cc:237] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:36.155874 11871 pending_rounds.cc:62] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade: Trying to abort 1 pending ops.
I20260926 07:35:36.156689 11311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52d064b684d0491e91f176d24eddabe1" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e"
I20260926 07:35:36.157318 11437 tablet_service.cc:1467] Processing CreateTablet for tablet eb9878f5b28f4d6589d838f70af464c8 (DEFAULT_TABLE table=default.Table_d6bf3b3f4b934ec093561293afadaea1 [id=d14935a6188c4e5387f0c79838d4f708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.158344 11437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb9878f5b28f4d6589d838f70af464c8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:36.160445 11401 leader_election.cc:341] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:36.127909 12259 ts_tablet_manager.cc:1918] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:36.163022 12259 log.cc:1196] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/4239f5d48512403bb711ece401f0756b
I20260926 07:35:36.163767 12259 ts_tablet_manager.cc:1939] T 4239f5d48512403bb711ece401f0756b P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:36.174188 12038 ts_manager.cc:194] Registered new tserver with Master: e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:36.174744 11289 tablet_service.cc:1467] Processing CreateTablet for tablet 33ca5d33a5184ce3b4fe88a9a6755124 (DEFAULT_TABLE table=default.Table_8d6e8e19741e40d0a01d30a0f21fef8a [id=cd0aa16a032f446b8cb1b5cc901b106e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.175768 11289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33ca5d33a5184ce3b4fe88a9a6755124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.180208 12304 leader_election.cc:288] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:36.181219 11871 pending_rounds.cc:68] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333511706634612736 op_type: NO_OP noop_request { }
I20260926 07:35:36.182087 11311 raft_consensus.cc:3037] T 7e52ec7d802a4e90bccef777f77a2a1b P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:36.182331 11871 raft_consensus.cc:2866] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 07:35:36.184080 11871 raft_consensus.cc:2258] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:36.190375 11871 ts_tablet_manager.cc:1905] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:36.159896 11754 ts_tablet_manager.cc:1428] T 25b8b701563344bbbc986db2b0f39b9b P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.064s user 0.003s sys 0.003s
I20260926 07:35:36.164829 11580 tablet_service.cc:1467] Processing CreateTablet for tablet eb9878f5b28f4d6589d838f70af464c8 (DEFAULT_TABLE table=default.Table_d6bf3b3f4b934ec093561293afadaea1 [id=d14935a6188c4e5387f0c79838d4f708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.168689 12354 raft_consensus.cc:482] T b8e926ee35854c28a8c698384526fe38 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:36.193122 12354 raft_consensus.cc:504] T b8e926ee35854c28a8c698384526fe38 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:36.193889 11746 consensus_queue.cc:260] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:36.195346 12354 leader_election.cc:288] T b8e926ee35854c28a8c698384526fe38 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:36.196882 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb9878f5b28f4d6589d838f70af464c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.198765 11111 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 0eae53ce278749208492815b918582f3 (table default.Table_56493b3efdaa43e684afcd3d4b5dffdd [id=f66bffa952604b28bb51299de795ee2a]). Expected version 1 got 0
I20260926 07:35:36.199753 11111 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet ae16af9edff9401589bcf54eeb3e8dfe (table default.Table_f9178d5e3d934f1f9ef267a782316458 [id=0c18461963f444269787b667b3648709]). Expected version 2 got 0
I20260926 07:35:36.200271 11111 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 0218d2a727f942e48d770878e6bf4a54 (table default.Table_e8b00b73be7c4818adf8d1e74d731ee6 [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b]). Expected version 3 got 0
I20260926 07:35:36.202904 11754 tablet_bootstrap.cc:492] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:36.206249 11442 tablet_service.cc:1467] Processing CreateTablet for tablet 33ca5d33a5184ce3b4fe88a9a6755124 (DEFAULT_TABLE table=default.Table_8d6e8e19741e40d0a01d30a0f21fef8a [id=cd0aa16a032f446b8cb1b5cc901b106e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.207126 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e926ee35854c28a8c698384526fe38" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:36.212227 11285 tablet_service.cc:1467] Processing CreateTablet for tablet 191f35545a13402db05b71dafa589845 (DEFAULT_TABLE table=default.Table_4c7ed90b32024fb888d7aef0e188ea80 [id=4e53b36526114bcabd8920c83fa2f6d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.213295 11285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 191f35545a13402db05b71dafa589845. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.226194 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e926ee35854c28a8c698384526fe38" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:36.226670 11756 tablet_bootstrap.cc:654] T 248ab84a1f284c2bbddf62f00b485e13 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:36.228587 11442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33ca5d33a5184ce3b4fe88a9a6755124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.270386 11437 tablet_service.cc:1467] Processing CreateTablet for tablet 191f35545a13402db05b71dafa589845 (DEFAULT_TABLE table=default.Table_4c7ed90b32024fb888d7aef0e188ea80 [id=4e53b36526114bcabd8920c83fa2f6d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.272279 11439 tablet_service.cc:1467] Processing CreateTablet for tablet f0e2d1ebf0ff4c57afb9b0b430c8ec21 (DEFAULT_TABLE table=default.Table_e888a0ff22884ed0bf88475c3f95b42c [id=34f36bd896994f91bccecb8b1e8995b7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 07:35:36.232842 11104 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.171.61:38823, user_credentials={real_user=slave}} blocked reactor thread for 82186.5us
W20260926 07:35:36.208684 11546 leader_election.cc:341] T b8e926ee35854c28a8c698384526fe38 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:36.274179 11284 tablet_service.cc:1467] Processing CreateTablet for tablet f0e2d1ebf0ff4c57afb9b0b430c8ec21 (DEFAULT_TABLE table=default.Table_e888a0ff22884ed0bf88475c3f95b42c [id=34f36bd896994f91bccecb8b1e8995b7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 07:35:36.274485 11546 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.1:41829, user_credentials={real_user=slave}} blocked reactor thread for 65841.2us
I20260926 07:35:36.275168 11284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0e2d1ebf0ff4c57afb9b0b430c8ec21. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:36.275651 11545 leader_election.cc:341] T b8e926ee35854c28a8c698384526fe38 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:36.212759 11581 tablet_service.cc:1467] Processing CreateTablet for tablet 33ca5d33a5184ce3b4fe88a9a6755124 (DEFAULT_TABLE table=default.Table_8d6e8e19741e40d0a01d30a0f21fef8a [id=cd0aa16a032f446b8cb1b5cc901b106e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.277275 11545 leader_election.cc:302] T b8e926ee35854c28a8c698384526fe38 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
W20260926 07:35:36.277981 11545 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.2:42379, user_credentials={real_user=slave}} blocked reactor thread for 44712us
I20260926 07:35:36.279279 11583 tablet_service.cc:1467] Processing CreateTablet for tablet 191f35545a13402db05b71dafa589845 (DEFAULT_TABLE table=default.Table_4c7ed90b32024fb888d7aef0e188ea80 [id=4e53b36526114bcabd8920c83fa2f6d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.279557 11581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33ca5d33a5184ce3b4fe88a9a6755124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.279644 11586 tablet_service.cc:1467] Processing CreateTablet for tablet f0e2d1ebf0ff4c57afb9b0b430c8ec21 (DEFAULT_TABLE table=default.Table_e888a0ff22884ed0bf88475c3f95b42c [id=34f36bd896994f91bccecb8b1e8995b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.283948 12259 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 161) took 2250 ms. Trace:
I20260926 07:35:36.285794 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 191f35545a13402db05b71dafa589845. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.288285 12354 raft_consensus.cc:2726] T b8e926ee35854c28a8c698384526fe38 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:36.288419 11586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0e2d1ebf0ff4c57afb9b0b430c8ec21. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:36.298622 11113 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 85f987ee219941cb824e0d68b5660b55 has no consensus state
I20260926 07:35:36.306068 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_fff130a3d2e2461ba2e11f11de45bdf4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:36.310335 11113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fff130a3d2e2461ba2e11f11de45bdf4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:36.326786 11754 tablet_bootstrap.cc:654] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:36.330633 11120 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 0218d2a727f942e48d770878e6bf4a54 (table default.Table_e8b00b73be7c4818adf8d1e74d731ee6 [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b]). Expected version 3 got 0
I20260926 07:35:36.331396 11120 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 0eae53ce278749208492815b918582f3 (table default.Table_56493b3efdaa43e684afcd3d4b5dffdd [id=f66bffa952604b28bb51299de795ee2a]). Expected version 1 got 0
I20260926 07:35:36.296399 12304 consensus_queue.cc:1035] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:36.322006 12292 consensus_queue.cc:1035] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 07:35:36.337708 11401 consensus_peers.cc:487] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:36.272295 11437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 191f35545a13402db05b71dafa589845. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.339385 11439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0e2d1ebf0ff4c57afb9b0b430c8ec21. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:36.332020 11104 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 85f987ee219941cb824e0d68b5660b55 (table default.Table_1330df49c5964507ba4c568038f33499 [id=73d12ba7ff404bdf9ad19c710094c872], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 85f987ee219941cb824e0d68b5660b55 has no consensus state
I20260926 07:35:36.353282 12310 raft_consensus.cc:482] T 04180ace5d4548dfbcd226bf229d5c26 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:36.195868 11746 ts_tablet_manager.cc:1428] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.076s user 0.004s sys 0.000s
I20260926 07:35:36.369725 11746 tablet_bootstrap.cc:492] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:36.388067 11113 master_service.cc:946] ReplaceTablet: received request to replace tablet 04180ace5d4548dfbcd226bf229d5c26 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:36.391258 11112 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet 0218d2a727f942e48d770878e6bf4a54 (table default.Table_e8b00b73be7c4818adf8d1e74d731ee6 [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b]). Expected version 3 got 0
I20260926 07:35:36.391991 11112 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet 0eae53ce278749208492815b918582f3 (table default.Table_56493b3efdaa43e684afcd3d4b5dffdd [id=f66bffa952604b28bb51299de795ee2a]). Expected version 1 got 0
I20260926 07:35:36.399435 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_665b626e1a0545cfbbb36ef06105dc06"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:36.401108 11113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_665b626e1a0545cfbbb36ef06105dc06 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:36.409396 11580 tablet_service.cc:1467] Processing CreateTablet for tablet 1f8bf1a16daa4874b6e0947c878ff4df (DEFAULT_TABLE table=default.Table_091d9aec1e664dc1aa7f5f1006b33eaf [id=fe0b9c05438a47acbe8fafd21c11b7d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.410720 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f8bf1a16daa4874b6e0947c878ff4df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.415129 11436 tablet_service.cc:1467] Processing CreateTablet for tablet 6461f74d89404b01967d7a6a5af6e284 (DEFAULT_TABLE table=default.Table_f0a09bb101f848819059089fd6a2aa65 [id=24ecefea9c5c41f394d01aa845194276]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.415133 11435 tablet_service.cc:1467] Processing CreateTablet for tablet 1f8bf1a16daa4874b6e0947c878ff4df (DEFAULT_TABLE table=default.Table_091d9aec1e664dc1aa7f5f1006b33eaf [id=fe0b9c05438a47acbe8fafd21c11b7d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.414381 11296 tablet_service.cc:1467] Processing CreateTablet for tablet 6461f74d89404b01967d7a6a5af6e284 (DEFAULT_TABLE table=default.Table_f0a09bb101f848819059089fd6a2aa65 [id=24ecefea9c5c41f394d01aa845194276]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.418473 11296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6461f74d89404b01967d7a6a5af6e284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.353745 12310 raft_consensus.cc:504] T 04180ace5d4548dfbcd226bf229d5c26 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:36.418769 11444 tablet_service.cc:1467] Processing CreateTablet for tablet ffb50e0189d64ddc9642037d87aed44c (DEFAULT_TABLE table=default.Table_2b10a1845a4044a8a8ecce2c10436ae6 [id=63fd52b5b7db426ebc9f7f2869d701e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.421083 11288 tablet_service.cc:1467] Processing CreateTablet for tablet ffb50e0189d64ddc9642037d87aed44c (DEFAULT_TABLE table=default.Table_2b10a1845a4044a8a8ecce2c10436ae6 [id=63fd52b5b7db426ebc9f7f2869d701e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.421682 12310 leader_election.cc:288] T 04180ace5d4548dfbcd226bf229d5c26 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:36.421020 11291 tablet_service.cc:1467] Processing CreateTablet for tablet 1f8bf1a16daa4874b6e0947c878ff4df (DEFAULT_TABLE table=default.Table_091d9aec1e664dc1aa7f5f1006b33eaf [id=fe0b9c05438a47acbe8fafd21c11b7d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.418780 11114 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 0eae53ce278749208492815b918582f3 (table default.Table_56493b3efdaa43e684afcd3d4b5dffdd [id=f66bffa952604b28bb51299de795ee2a]). Expected version 1 got 0
I20260926 07:35:36.422792 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04180ace5d4548dfbcd226bf229d5c26" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:36.423369 11591 tablet_service.cc:1467] Processing CreateTablet for tablet ffb50e0189d64ddc9642037d87aed44c (DEFAULT_TABLE table=default.Table_2b10a1845a4044a8a8ecce2c10436ae6 [id=63fd52b5b7db426ebc9f7f2869d701e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.424388 11591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb50e0189d64ddc9642037d87aed44c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.407933 11583 tablet_service.cc:1467] Processing CreateTablet for tablet 6461f74d89404b01967d7a6a5af6e284 (DEFAULT_TABLE table=default.Table_f0a09bb101f848819059089fd6a2aa65 [id=24ecefea9c5c41f394d01aa845194276]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.425558 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6461f74d89404b01967d7a6a5af6e284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.424418 11114 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 0218d2a727f942e48d770878e6bf4a54 (table default.Table_e8b00b73be7c4818adf8d1e74d731ee6 [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b]). Expected version 3 got 0
I20260926 07:35:36.427783 11114 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 248ab84a1f284c2bbddf62f00b485e13 (table default.Table_8439aa0b11574c5b822c46f01f75d842 [id=47d4200172794cc78a6f386fee4b15b4]). Expected version 3 got 0
I20260926 07:35:36.422056 11288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb50e0189d64ddc9642037d87aed44c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.435146 11746 tablet_bootstrap.cc:654] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:36.435422 11871 ts_tablet_manager.cc:1918] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:36.435803 11871 log.cc:1196] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/034d508f0a164f3ea8308ea2d28818c4
I20260926 07:35:36.436648 11871 ts_tablet_manager.cc:1939] T 034d508f0a164f3ea8308ea2d28818c4 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:36.448999 11588 tablet_service.cc:1467] Processing CreateTablet for tablet 384d960c8b714d30b0f21a179342b0ce (DEFAULT_TABLE table=default.Table_1887d867f46048ed8cdb47823c34d138 [id=1e08bc78b765441f90ae233d6d1c62e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.449950 11588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 384d960c8b714d30b0f21a179342b0ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.453114 11582 tablet_service.cc:1467] Processing CreateTablet for tablet 41851e8f88dd473a880242a7fabd2a92 (DEFAULT_TABLE table=default.Table_bd94d128c83c44a3bcc621acb91820c0 [id=60287ba286274f659a2dee13db5f42d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.454093 11582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41851e8f88dd473a880242a7fabd2a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.376801 12259 rpcz_store.cc:276] 0926 07:35:34.033778 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.033875 (+ 97us) service_pool.cc:227] Handling call
0926 07:35:35.901537 (+1867662us) ts_tablet_manager.cc:1145] Deleting tablet 4239f5d48512403bb711ece401f0756b
0926 07:35:35.901581 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:36.061326 (+159745us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:36.127696 (+ 66370us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:36.283890 (+156194us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1867225}
I20260926 07:35:36.446305 11440 tablet_service.cc:1467] Processing CreateTablet for tablet 384d960c8b714d30b0f21a179342b0ce (DEFAULT_TABLE table=default.Table_1887d867f46048ed8cdb47823c34d138 [id=1e08bc78b765441f90ae233d6d1c62e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.450537 11445 tablet_service.cc:1467] Processing CreateTablet for tablet 41851e8f88dd473a880242a7fabd2a92 (DEFAULT_TABLE table=default.Table_bd94d128c83c44a3bcc621acb91820c0 [id=60287ba286274f659a2dee13db5f42d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.473425 11113 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_ec15d379852d4e8a8b80c63122bf3bc7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:36.476406 11113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ec15d379852d4e8a8b80c63122bf3bc7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:36.480829 11287 tablet_service.cc:1467] Processing CreateTablet for tablet 384d960c8b714d30b0f21a179342b0ce (DEFAULT_TABLE table=default.Table_1887d867f46048ed8cdb47823c34d138 [id=1e08bc78b765441f90ae233d6d1c62e3]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 07:35:36.443670 11249 leader_election.cc:341] T 04180ace5d4548dfbcd226bf229d5c26 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Not found: Tablet not found: 04180ace5d4548dfbcd226bf229d5c26
I20260926 07:35:36.490563 11290 tablet_service.cc:1467] Processing CreateTablet for tablet 41851e8f88dd473a880242a7fabd2a92 (DEFAULT_TABLE table=default.Table_bd94d128c83c44a3bcc621acb91820c0 [id=60287ba286274f659a2dee13db5f42d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.492477 11291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f8bf1a16daa4874b6e0947c878ff4df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.492358 11249 leader_election.cc:302] T 04180ace5d4548dfbcd226bf229d5c26 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9bfebb34fa14cd2ad4843fffbe5c54e; no voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade
W20260926 07:35:36.496975 11249 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.3:45127, user_credentials={real_user=slave}} blocked reactor thread for 59458.7us
I20260926 07:35:36.497502 12310 raft_consensus.cc:2726] T 04180ace5d4548dfbcd226bf229d5c26 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:36.496745 11118 catalog_manager.cc:5495] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22f3902d01584936b20437606cb86dd1 (127.10.171.2). New cstate: current_term: 1 leader_uuid: "22f3902d01584936b20437606cb86dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260926 07:35:36.498704 11290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41851e8f88dd473a880242a7fabd2a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.499243 11118 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 0eae53ce278749208492815b918582f3 (table default.Table_56493b3efdaa43e684afcd3d4b5dffdd [id=f66bffa952604b28bb51299de795ee2a]). Expected version 1 got 0
I20260926 07:35:36.504498 11287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 384d960c8b714d30b0f21a179342b0ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.520787 11746 tablet_bootstrap.cc:492] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:36.521199 11746 ts_tablet_manager.cc:1397] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.152s user 0.007s sys 0.002s
I20260926 07:35:36.523307 11746 raft_consensus.cc:348] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:36.528435 12168 ts_tablet_manager.cc:1918] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260926 07:35:36.529145 12168 log.cc:1196] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/a0bdf6a4656e4217b0a7013fddc4cca8
I20260926 07:35:36.529806 12168 ts_tablet_manager.cc:1939] T a0bdf6a4656e4217b0a7013fddc4cca8 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:36.523924 11746 raft_consensus.cc:374] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:36.539556 11746 raft_consensus.cc:729] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:36.540184 11746 consensus_queue.cc:260] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:36.541494 12168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 149) took 2837 ms. Trace:
I20260926 07:35:36.541831 12168 rpcz_store.cc:276] 0926 07:35:33.704447 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:33.704624 (+ 177us) service_pool.cc:227] Handling call
0926 07:35:35.921492 (+2216868us) ts_tablet_manager.cc:1145] Deleting tablet a0bdf6a4656e4217b0a7013fddc4cca8
0926 07:35:35.921530 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:36.027000 (+105470us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:36.528228 (+501228us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:36.541448 (+ 13220us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2160345}
I20260926 07:35:36.543417 12168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 153) took 2837 ms. Trace:
I20260926 07:35:36.543747 12168 rpcz_store.cc:276] 0926 07:35:33.705923 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:33.708065 (+ 2142us) service_pool.cc:227] Handling call
0926 07:35:36.543117 (+2835052us) ts_tablet_manager.cc:1145] Deleting tablet 7019af1230224e999e078b21ea5a30a5
0926 07:35:36.543154 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:36.543374 (+ 220us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2779760}
I20260926 07:35:36.557967 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 138) took 4816 ms. Trace:
I20260926 07:35:36.558379 11871 rpcz_store.cc:276] 0926 07:35:31.741201 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:31.741332 (+ 131us) service_pool.cc:227] Handling call
0926 07:35:36.104494 (+4363162us) ts_tablet_manager.cc:1145] Deleting tablet 034d508f0a164f3ea8308ea2d28818c4
0926 07:35:36.104532 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:36.270992 (+166460us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:36.435224 (+164232us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:36.557911 (+122687us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4362649}
I20260926 07:35:36.562199 11746 ts_tablet_manager.cc:1428] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.041s user 0.007s sys 0.000s
I20260926 07:35:36.564472 12168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 158) took 2561 ms. Trace:
I20260926 07:35:36.564791 12168 rpcz_store.cc:276] 0926 07:35:34.003250 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.077058 (+ 73808us) service_pool.cc:227] Handling call
0926 07:35:36.544838 (+2467780us) ts_tablet_manager.cc:1145] Deleting tablet 640d21586bad4070957cff61bd174fd2
0926 07:35:36.544883 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:36.564415 (+ 19532us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2466571}
I20260926 07:35:36.564417 11118 catalog_manager.cc:5495] T 0f20206ce6fd4fd395f1b2b0154245db P 22f3902d01584936b20437606cb86dd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22f3902d01584936b20437606cb86dd1 (127.10.171.2). New cstate: current_term: 1 leader_uuid: "22f3902d01584936b20437606cb86dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260926 07:35:36.566197 11118 catalog_manager.cc:5495] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22f3902d01584936b20437606cb86dd1 (127.10.171.2). New cstate: current_term: 1 leader_uuid: "22f3902d01584936b20437606cb86dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260926 07:35:36.543028 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet a0bdf6a4656e4217b0a7013fddc4cca8 (table default.Table_1da003049e4b46eb994aaa3f3094d63c [id=3ce63f9fb5ad4b82ba2fbbf2e3678271]) successfully deleted
W20260926 07:35:36.568933 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 7019af1230224e999e078b21ea5a30a5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7019af1230224e999e078b21ea5a30a5 already in progress: creating tablet
W20260926 07:35:36.569470 11106 catalog_manager.cc:4808] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 640d21586bad4070957cff61bd174fd2 because the tablet was not found. No further retry: Not found: Tablet not found: 640d21586bad4070957cff61bd174fd2
I20260926 07:35:36.579021 11590 tablet_service.cc:1467] Processing CreateTablet for tablet 76eb30d364784ec481e11585bd93446e (DEFAULT_TABLE table=default.Table_7941154c519c4b23a2983c2b3ec82ec2 [id=db4d15f27d9e4c6bb4e0b3f52959b882]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.577332 11586 tablet_service.cc:1467] Processing CreateTablet for tablet 089bac5feeaa4508bc863e446b172033 (DEFAULT_TABLE table=default.Table_1da003049e4b46eb994aaa3f3094d63c [id=3ce63f9fb5ad4b82ba2fbbf2e3678271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.584908 11581 tablet_service.cc:1467] Processing CreateTablet for tablet 9be5c479f2544d2dac7da0b311083dcf (DEFAULT_TABLE table=default.Table_597d2bba0c2b4bfc8c1e14e454b61b86 [id=a28d622db9cf420faa75d9aa65e9bcae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.560101 11871 tablet_replica.cc:330] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:36.615262 11871 raft_consensus.cc:2229] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:36.615882 11871 raft_consensus.cc:2258] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:36.617653 11590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76eb30d364784ec481e11585bd93446e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.622036 11586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 089bac5feeaa4508bc863e446b172033. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.592540 11285 tablet_service.cc:1467] Processing CreateTablet for tablet 089bac5feeaa4508bc863e446b172033 (DEFAULT_TABLE table=default.Table_1da003049e4b46eb994aaa3f3094d63c [id=3ce63f9fb5ad4b82ba2fbbf2e3678271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.632826 11581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9be5c479f2544d2dac7da0b311083dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.634241 12168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 157) took 2631 ms. Trace:
I20260926 07:35:36.634683 12168 rpcz_store.cc:276] 0926 07:35:34.002972 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.076585 (+ 73613us) service_pool.cc:227] Handling call
0926 07:35:36.565902 (+2489317us) ts_tablet_manager.cc:1145] Deleting tablet 05f071b4db7847a5adbd25a71cac3ea7
0926 07:35:36.565947 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:36.634172 (+ 68225us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2487555}
W20260926 07:35:36.635871 11106 catalog_manager.cc:4808] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 05f071b4db7847a5adbd25a71cac3ea7 because the tablet was not found. No further retry: Not found: Tablet not found: 05f071b4db7847a5adbd25a71cac3ea7
I20260926 07:35:36.592753 11295 tablet_service.cc:1467] Processing CreateTablet for tablet 9be5c479f2544d2dac7da0b311083dcf (DEFAULT_TABLE table=default.Table_597d2bba0c2b4bfc8c1e14e454b61b86 [id=a28d622db9cf420faa75d9aa65e9bcae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.649115 11294 tablet_service.cc:1467] Processing CreateTablet for tablet 76eb30d364784ec481e11585bd93446e (DEFAULT_TABLE table=default.Table_7941154c519c4b23a2983c2b3ec82ec2 [id=db4d15f27d9e4c6bb4e0b3f52959b882]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:36.567957 11118 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 52d064b684d0491e91f176d24eddabe1 (table default.Table_e9b22035d94d423280e51a4864e4aa84 [id=d19e4ba11fa14fa18c7ae93bf40f7c89]). Expected version 2 got 0
I20260926 07:35:36.595397 11754 tablet_bootstrap.cc:492] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:36.669297 11118 catalog_manager.cc:5495] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22f3902d01584936b20437606cb86dd1 (127.10.171.2). New cstate: current_term: 1 leader_uuid: "22f3902d01584936b20437606cb86dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } health_report { overall_health: UNKNOWN } } }
I20260926 07:35:36.671008 11754 ts_tablet_manager.cc:1397] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.469s user 0.005s sys 0.005s
I20260926 07:35:36.690410 11756 tablet_bootstrap.cc:492] T 248ab84a1f284c2bbddf62f00b485e13 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:36.690824 11756 ts_tablet_manager.cc:1397] T 248ab84a1f284c2bbddf62f00b485e13 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.638s user 0.007s sys 0.006s
I20260926 07:35:36.698748 11754 raft_consensus.cc:348] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:36.712678 11118 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 248ab84a1f284c2bbddf62f00b485e13 (table default.Table_8439aa0b11574c5b822c46f01f75d842 [id=47d4200172794cc78a6f386fee4b15b4]). Expected version 3 got 0
I20260926 07:35:36.707708 12133 raft_consensus.cc:482] T 25b8b701563344bbbc986db2b0f39b9b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:36.725996 12133 raft_consensus.cc:504] T 25b8b701563344bbbc986db2b0f39b9b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:36.727927 12133 leader_election.cc:288] T 25b8b701563344bbbc986db2b0f39b9b P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:36.682189 11871 ts_tablet_manager.cc:1905] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:36.731675 11756 raft_consensus.cc:348] T 248ab84a1f284c2bbddf62f00b485e13 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:36.732723 11756 raft_consensus.cc:374] T 248ab84a1f284c2bbddf62f00b485e13 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:36.733167 11756 raft_consensus.cc:729] T 248ab84a1f284c2bbddf62f00b485e13 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:36.740254 11436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6461f74d89404b01967d7a6a5af6e284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.747530 11754 raft_consensus.cc:374] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:36.748857 11754 raft_consensus.cc:729] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:36.750655 11754 consensus_queue.cc:260] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:36.754866 11754 ts_tablet_manager.cc:1428] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.058s user 0.004s sys 0.001s
I20260926 07:35:36.757014 11435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f8bf1a16daa4874b6e0947c878ff4df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.761969 11440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 384d960c8b714d30b0f21a179342b0ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.766014 11444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb50e0189d64ddc9642037d87aed44c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.768877 11445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41851e8f88dd473a880242a7fabd2a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.799060 11113 catalog_manager.cc:5587] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829) does not have the latest schema for tablet 0eae53ce278749208492815b918582f3 (table default.Table_56493b3efdaa43e684afcd3d4b5dffdd [id=f66bffa952604b28bb51299de795ee2a]). Expected version 1 got 0
I20260926 07:35:36.808465 10924 master-stress-test.cc:578] Killing test thread 1/10
I20260926 07:35:36.733968 11756 consensus_queue.cc:260] T 248ab84a1f284c2bbddf62f00b485e13 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:36.819953 11119 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: default.Table_f9178d5e3d934f1f9ef267a782316458 [id=0c18461963f444269787b667b3648709] alter complete (version 2)
I20260926 07:35:36.822888 11115 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:44836 (request call id 274) took 1290 ms. Trace:
I20260926 07:35:36.823176 11115 rpcz_store.cc:276] 0926 07:35:35.532317 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:35.532720 (+ 403us) service_pool.cc:227] Handling call
0926 07:35:35.533579 (+ 859us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0926 07:35:35.534214 (+ 635us) catalog_manager.cc:3616] Apply alter schema
0926 07:35:35.534716 (+ 502us) catalog_manager.cc:3639] Acquired catalog manager lock
0926 07:35:35.534900 (+ 184us) catalog_manager.cc:3786] Updating metadata on disk
0926 07:35:35.617103 (+ 82203us) catalog_manager.cc:3817] Committing alterations to in-memory state
0926 07:35:36.822833 (+1205730us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:35.563352 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:35.563697 (+ 345us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:35.563721 (+ 24us) write_op.cc:435] Acquired schema lock
0926 07:35:35.563735 (+ 14us) tablet.cc:660] Decoding operations
0926 07:35:35.563964 (+ 229us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:35.563994 (+ 30us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:35.564015 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0926 07:35:35.565772 (+ 1757us) tablet.cc:699] Row locks acquired
0926 07:35:35.565792 (+ 20us) write_op.cc:260] PREPARE: finished
0926 07:35:35.565908 (+ 116us) write_op.cc:270] Start()
0926 07:35:35.565995 (+ 87us) write_op.cc:276] Timestamp: P: 1790408135565843 usec, L: 0
0926 07:35:35.566014 (+ 19us) op_driver.cc:331] REPLICATION: starting
0926 07:35:35.566495 (+ 481us) log.cc:842] Serialized 405 byte log entry
0926 07:35:35.607519 (+ 41024us) spinlock_profiling.cc:243] Waited 8.01 ms on lock 0x7b5400012320. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb810336028 00007fb810335c6a 00007fb8103a1cd5 00007fb8103a2179 00007fb810803ade 00007fb8108032b8 00007fb810807ba1 00007fb810807b59 00007fb810807ae9 00007fb810807ab1
0926 07:35:35.612446 (+ 4927us) op_driver.cc:444] REPLICATION: finished
0926 07:35:35.615761 (+ 3315us) write_op.cc:301] APPLY: starting
0926 07:35:35.615810 (+ 49us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:35.615904 (+ 94us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:35.615926 (+ 22us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:35.616068 (+ 142us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:35.616207 (+ 139us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:35.616252 (+ 45us) write_op.cc:312] APPLY: finished
0926 07:35:35.616392 (+ 140us) log.cc:842] Serialized 23 byte log entry
0926 07:35:35.616507 (+ 115us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:35.616555 (+ 48us) write_op.cc:454] Released schema lock
0926 07:35:35.616585 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:35.616723 (+ 138us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:35.616747 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3186,"apply.run_cpu_time_us":1108,"apply.run_wall_time_us":1108,"mutex_wait_us":21891,"num_ops":1,"prepare.queue_time_us":27519,"prepare.run_cpu_time_us":3365,"prepare.run_wall_time_us":44304,"replication_time_us":46362,"spinlock_wait_cycles":18434304}]]}
I20260926 07:35:36.857317 11116 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 2dd5175f0ca941f8b3856ea415e2ee68
I20260926 07:35:36.858696 11116 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:44836 (request call id 267) took 1576 ms. Trace:
I20260926 07:35:36.859014 11116 rpcz_store.cc:276] 0926 07:35:35.282429 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:35.282606 (+ 177us) service_pool.cc:227] Handling call
0926 07:35:35.297591 (+ 14985us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812376466 00007fb80feb7b34 00007fb80feb7ac2 00007fb80feb79f1 00007fb80feb796c 00007fb80feb6c22 00007fb80e15594c 00007fb80e154d86
0926 07:35:35.298484 (+ 893us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0926 07:35:35.298955 (+ 471us) catalog_manager.cc:2752] Modifying in-memory table state
0926 07:35:35.299086 (+ 131us) catalog_manager.cc:2761] Locking tablets
0926 07:35:36.066160 (+767074us) catalog_manager.cc:2775] Removing table and tablets from system table
0926 07:35:36.162176 (+ 96016us) catalog_manager.cc:2793] Removing table from by-name map
0926 07:35:36.162416 (+ 240us) catalog_manager.cc:2809] Committing in-memory state
0926 07:35:36.857221 (+694805us) catalog_manager.cc:2813] Aborting table tasks
0926 07:35:36.858643 (+ 1422us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:36.076419 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:36.076713 (+ 294us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:36.076733 (+ 20us) write_op.cc:435] Acquired schema lock
0926 07:35:36.076746 (+ 13us) tablet.cc:660] Decoding operations
0926 07:35:36.076999 (+ 253us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:36.077030 (+ 31us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:36.077049 (+ 19us) tablet.cc:683] Acquiring locks for 2 operations
0926 07:35:36.077277 (+ 228us) tablet.cc:699] Row locks acquired
0926 07:35:36.077290 (+ 13us) write_op.cc:260] PREPARE: finished
0926 07:35:36.077386 (+ 96us) write_op.cc:270] Start()
0926 07:35:36.077451 (+ 65us) write_op.cc:276] Timestamp: P: 1790408136077313 usec, L: 0
0926 07:35:36.077466 (+ 15us) op_driver.cc:331] REPLICATION: starting
0926 07:35:36.077838 (+ 372us) log.cc:842] Serialized 759 byte log entry
0926 07:35:36.159896 (+ 82058us) op_driver.cc:444] REPLICATION: finished
0926 07:35:36.160689 (+ 793us) write_op.cc:301] APPLY: starting
0926 07:35:36.160747 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:36.160846 (+ 99us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:36.160931 (+ 85us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:36.161173 (+ 242us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:36.161346 (+ 173us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0926 07:35:36.161418 (+ 72us) write_op.cc:312] APPLY: finished
0926 07:35:36.161623 (+ 205us) log.cc:842] Serialized 29 byte log entry
0926 07:35:36.161759 (+ 136us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:36.161819 (+ 60us) write_op.cc:454] Released schema lock
0926 07:35:36.161847 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:36.162005 (+ 158us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:36.162027 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":33075584,"child_traces":[["op",{"apply.queue_time_us":648,"apply.run_cpu_time_us":1504,"apply.run_wall_time_us":1503,"num_ops":2,"prepare.queue_time_us":8745,"prepare.run_cpu_time_us":1576,"prepare.run_wall_time_us":1575,"replication_time_us":82361}]]}
I20260926 07:35:36.862355 11116 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_597d2bba0c2b4bfc8c1e14e454b61b86" } modify_external_catalogs: true
I20260926 07:35:36.862839 11116 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_597d2bba0c2b4bfc8c1e14e454b61b86" } modify_external_catalogs: true
I20260926 07:35:36.861716 11754 tablet_bootstrap.cc:492] T 6c42b518ef04429aa6b4e8f5deee05c8 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:36.866124 11580 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dd5175f0ca941f8b3856ea415e2ee68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:35 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:36.866240 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25b8b701563344bbbc986db2b0f39b9b" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
W20260926 07:35:36.868388 11546 leader_election.cc:341] T 25b8b701563344bbbc986db2b0f39b9b P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:36.872941 11756 ts_tablet_manager.cc:1428] T 248ab84a1f284c2bbddf62f00b485e13 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.182s user 0.009s sys 0.000s
I20260926 07:35:36.882730 11582 tablet_service.cc:1514] Processing DeleteTablet for tablet 04180ace5d4548dfbcd226bf229d5c26 with delete_type TABLET_DATA_DELETED (replaced by tablet b862f325a9df49079066dc62fe1a61f1) from {username='slave'} at 127.0.0.1:35244
W20260926 07:35:36.884047 11111 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.3:53335 (request call id 32) took 2209 ms (client timeout 1000 ms). Trace:
W20260926 07:35:36.884397 11111 rpcz_store.cc:269] 0926 07:35:34.675125 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.763868 (+ 88743us) service_pool.cc:227] Handling call
0926 07:35:35.107363 (+343495us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812374c3e 00007fb80feabfb4 00007fb80feabf42 00007fb80feabe71 00007fb80feabdec 00007fb80feab0a2 00007fb80e15594c 00007fb80e154d86
0926 07:35:36.883984 (+1776621us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":10497,"reported_tablets":27,"spinlock_wait_cycles":41832832}
I20260926 07:35:36.886430 11111 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_6336482092624a0b849ec138f2e928bd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:36.887818 11111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6336482092624a0b849ec138f2e928bd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:36.890007 12168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 161) took 2807 ms. Trace:
I20260926 07:35:36.890290 12168 rpcz_store.cc:276] 0926 07:35:34.082303 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.082714 (+ 411us) service_pool.cc:227] Handling call
0926 07:35:36.636200 (+2553486us) ts_tablet_manager.cc:1145] Deleting tablet 034d508f0a164f3ea8308ea2d28818c4
0926 07:35:36.889804 (+253604us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:36.889958 (+ 154us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2547351}
I20260926 07:35:36.891201 12168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 159) took 2815 ms. Trace:
I20260926 07:35:36.891445 12168 rpcz_store.cc:276] 0926 07:35:34.076209 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.088263 (+ 12054us) service_pool.cc:227] Handling call
0926 07:35:36.890979 (+2802716us) ts_tablet_manager.cc:1145] Deleting tablet d6fc6178f1e54c3984a20873864233cd
0926 07:35:36.891013 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:36.891158 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2802050}
I20260926 07:35:36.892329 11590 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e28ca159d8349748812825ac1947c51 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dd5175f0ca941f8b3856ea415e2ee68) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:36.893114 12168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 156) took 2913 ms. Trace:
I20260926 07:35:36.893435 12168 rpcz_store.cc:276] 0926 07:35:34.002451 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.076628 (+ 74177us) service_pool.cc:227] Handling call
0926 07:35:36.892810 (+2816182us) ts_tablet_manager.cc:1145] Deleting tablet 4088ccb81e844cc29e1735bb341e4a92
0926 07:35:36.892849 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:36.893060 (+ 211us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2802592}
I20260926 07:35:36.895192 12168 tablet_replica.cc:330] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:36.895139 11586 tablet_service.cc:1514] Processing DeleteTablet for tablet a000cb9928de4459be1e6b1b7a81848a with delete_type TABLET_DATA_DELETED (replaced by tablet 89ddc16f2635473da43fe21676a947fa) from {username='slave'} at 127.0.0.1:35244
W20260926 07:35:36.896670 11117 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.2:33317 (request call id 49) took 2957 ms (client timeout 1000 ms). Trace:
I20260926 07:35:36.897331 12168 raft_consensus.cc:2229] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:36.897733 12168 raft_consensus.cc:2258] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:36.899282 12168 ts_tablet_manager.cc:1905] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 07:35:36.896973 11117 rpcz_store.cc:269] 0926 07:35:33.939745 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.105629 (+165884us) service_pool.cc:227] Handling call
0926 07:35:34.347105 (+241476us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812374c3e 00007fb80feabfb4 00007fb80feabf42 00007fb80feabe71 00007fb80feabdec 00007fb80feab0a2 00007fb80e15594c 00007fb80e154d86
0926 07:35:36.896611 (+2549506us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:35.102121 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:35.102457 (+ 336us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:35.102484 (+ 27us) write_op.cc:435] Acquired schema lock
0926 07:35:35.102497 (+ 13us) tablet.cc:660] Decoding operations
0926 07:35:35.105238 (+ 2741us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:35.105271 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:35.105290 (+ 19us) tablet.cc:683] Acquiring locks for 1 operations
0926 07:35:35.105409 (+ 119us) tablet.cc:699] Row locks acquired
0926 07:35:35.105433 (+ 24us) write_op.cc:260] PREPARE: finished
0926 07:35:35.105517 (+ 84us) write_op.cc:270] Start()
0926 07:35:35.105588 (+ 71us) write_op.cc:276] Timestamp: P: 1790408135105436 usec, L: 0
0926 07:35:35.105603 (+ 15us) op_driver.cc:331] REPLICATION: starting
0926 07:35:35.142746 (+ 37143us) log.cc:842] Serialized 530 byte log entry
0926 07:35:35.148168 (+ 5422us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b5400012320. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032b471 00007fb81033f416 00007fb810338daa 00007fb810338014 00007fb810320545 00007fb810325771 00007fb8103256f9 00007fb810325689 00007fb810325651 00007fb81032481d 0000000000580784
0926 07:35:35.186745 (+ 38577us) op_driver.cc:444] REPLICATION: finished
0926 07:35:35.187212 (+ 467us) write_op.cc:301] APPLY: starting
0926 07:35:35.187262 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:35.187356 (+ 94us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:35.187374 (+ 18us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:35.187544 (+ 170us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:35.187674 (+ 130us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:35.187718 (+ 44us) write_op.cc:312] APPLY: finished
0926 07:35:35.187886 (+ 168us) log.cc:842] Serialized 23 byte log entry
0926 07:35:35.188525 (+ 639us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:35.188594 (+ 69us) write_op.cc:454] Released schema lock
0926 07:35:35.188632 (+ 38us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:35.188829 (+ 197us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:35.188861 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":483,"reported_tablets":18,"spinlock_wait_cycles":44168064,"child_traces":[["op",{"apply.queue_time_us":328,"apply.run_cpu_time_us":1344,"apply.run_wall_time_us":1809,"num_ops":1,"prepare.queue_time_us":100365,"prepare.run_cpu_time_us":1845,"prepare.run_wall_time_us":88696,"raft.queue_time_us":641,"raft.run_cpu_time_us":2005,"raft.run_wall_time_us":4813,"replication_time_us":81078,"spinlock_wait_cycles":5096704}]]}
W20260926 07:35:36.902904 11120 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.2:33317 (request call id 50) took 1782 ms (client timeout 1000 ms). Trace:
I20260926 07:35:36.903326 11756 tablet_bootstrap.cc:492] T dbd4f14abfea4799b75c48531412fc20 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:36.904099 11580 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e1cea4523a64428aac03ca0bdf708c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:24 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:36.905367 11294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76eb30d364784ec481e11585bd93446e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.906337 11295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9be5c479f2544d2dac7da0b311083dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.914636 11285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 089bac5feeaa4508bc863e446b172033. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:36.922505 11756 tablet_bootstrap.cc:654] T dbd4f14abfea4799b75c48531412fc20 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:36.922384 11120 rpcz_store.cc:269] 0926 07:35:35.120581 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:35.132791 (+ 12210us) service_pool.cc:227] Handling call
0926 07:35:35.167725 (+ 34934us) spinlock_profiling.cc:243] Waited 8.33 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812374c3e 00007fb80feabfb4 00007fb80feabf42 00007fb80feabe71 00007fb80feabdec 00007fb80feab0a2 00007fb80e15594c 00007fb80e154d86
0926 07:35:36.902849 (+1735124us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4,"reported_tablets":21,"spinlock_wait_cycles":19158912}
I20260926 07:35:36.903422 11117 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44836:
table { table_name: "default.Table_170efaa4a249400e8c1d772a5ac0ec54" } new_table_name: "default.Table_b61b46186a5a49bdba86babfd064d3ab" modify_external_catalogs: true
W20260926 07:35:36.910777 11114 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.3:53335 (request call id 33) took 1172 ms (client timeout 1000 ms). Trace:
W20260926 07:35:36.923792 11114 rpcz_store.cc:269] 0926 07:35:35.738436 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:36.012084 (+273648us) service_pool.cc:227] Handling call
0926 07:35:36.910716 (+898632us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4027,"reported_tablets":30}
I20260926 07:35:36.931149 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet 034d508f0a164f3ea8308ea2d28818c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 4088ccb81e844cc29e1735bb341e4a92) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:36.932770 11114 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_74460676db784adf8c5d5ff8d76a021c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:36.934237 11114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_74460676db784adf8c5d5ff8d76a021c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:36.935487 11120 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_a178527b84ff446e95c6439e43798348"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:35:36.941751 11291 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dd5175f0ca941f8b3856ea415e2ee68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:35 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:36.944911 12259 tablet_replica.cc:330] T c36c759963cb4915afdca7d12109cc6e P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:36.973541 11114 master_service.cc:946] ReplaceTablet: received request to replace tablet 2dd5175f0ca941f8b3856ea415e2ee68 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:36.976720 11746 tablet_bootstrap.cc:492] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:36.978142 11114 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44836:
name: "default.Table_779bd60d3ede453ca6d35ead529d31fc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:35:36.979635 11114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_779bd60d3ede453ca6d35ead529d31fc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:35:36.987118 11106 catalog_manager.cc:4808] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 034d508f0a164f3ea8308ea2d28818c4 because the tablet was not found. No further retry: Not found: Tablet not found: 034d508f0a164f3ea8308ea2d28818c4
W20260926 07:35:36.987701 11106 catalog_manager.cc:4808] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet d6fc6178f1e54c3984a20873864233cd because the tablet was not found. No further retry: Not found: Tablet not found: d6fc6178f1e54c3984a20873864233cd
W20260926 07:35:36.988955 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 4088ccb81e844cc29e1735bb341e4a92 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4088ccb81e844cc29e1735bb341e4a92 already in progress: creating tablet
I20260926 07:35:36.991261 11746 tablet_bootstrap.cc:654] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:36.997520 11116 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 9be5c479f2544d2dac7da0b311083dcf
I20260926 07:35:37.009752 11756 tablet_bootstrap.cc:492] T dbd4f14abfea4799b75c48531412fc20 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:37.010294 11756 ts_tablet_manager.cc:1397] T dbd4f14abfea4799b75c48531412fc20 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.107s user 0.009s sys 0.001s
I20260926 07:35:37.012539 11756 raft_consensus.cc:348] T dbd4f14abfea4799b75c48531412fc20 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.020828 10924 master-stress-test.cc:580] Killed test thread 1/10
I20260926 07:35:37.021087 10924 master-stress-test.cc:578] Killing test thread 2/10
I20260926 07:35:37.021080 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet 9be5c479f2544d2dac7da0b311083dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:36 UTC) from {username='slave'} at 127.0.0.1:35244
W20260926 07:35:37.043675 11401 consensus_peers.cc:487] T dbd4f14abfea4799b75c48531412fc20 P 22f3902d01584936b20437606cb86dd1 -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260926 07:35:37.043895 11546 consensus_peers.cc:487] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260926 07:35:37.044739 11401 consensus_peers.cc:487] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:37.055158 12259 raft_consensus.cc:2229] T c36c759963cb4915afdca7d12109cc6e P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:37.070925 12394 tablet.cc:1722] T a55e54d492214039b0294f954b5ef390 P 6070e9e3d1234a9d83e49431416afade: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260926 07:35:37.071466 11117 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f7459bf97b7f4b648d8ca6218cd72914 consensus state has no leader
I20260926 07:35:37.073324 12259 raft_consensus.cc:2258] T c36c759963cb4915afdca7d12109cc6e P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:37.075353 12259 ts_tablet_manager.cc:1905] T c36c759963cb4915afdca7d12109cc6e P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:37.028266 12385 raft_consensus.cc:482] T 248ab84a1f284c2bbddf62f00b485e13 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:37.080837 12385 raft_consensus.cc:504] T 248ab84a1f284c2bbddf62f00b485e13 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:37.084048 11607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248ab84a1f284c2bbddf62f00b485e13" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:37.084849 11607 raft_consensus.cc:2379] T 248ab84a1f284c2bbddf62f00b485e13 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9bfebb34fa14cd2ad4843fffbe5c54e in current term 1: Already voted for candidate 22f3902d01584936b20437606cb86dd1 in this term.
I20260926 07:35:37.086019 12385 leader_election.cc:288] T 248ab84a1f284c2bbddf62f00b485e13 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:37.044531 11296 tablet_service.cc:1514] Processing DeleteTablet for tablet 9be5c479f2544d2dac7da0b311083dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:36 UTC) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:37.069667 12393 tablet.cc:1722] T a55e54d492214039b0294f954b5ef390 P e9bfebb34fa14cd2ad4843fffbe5c54e: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 07:35:37.080329 11311 raft_consensus.cc:3037] T 04180ace5d4548dfbcd226bf229d5c26 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:37.093148 11756 raft_consensus.cc:374] T dbd4f14abfea4799b75c48531412fc20 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:37.093448 11756 raft_consensus.cc:729] T dbd4f14abfea4799b75c48531412fc20 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:37.094080 11756 consensus_queue.cc:260] T dbd4f14abfea4799b75c48531412fc20 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.099341 11756 ts_tablet_manager.cc:1428] T dbd4f14abfea4799b75c48531412fc20 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.089s user 0.005s sys 0.000s
I20260926 07:35:37.100230 11756 tablet_bootstrap.cc:492] T 3f581510fe624127a4d27d302f5be614 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
W20260926 07:35:37.102937 11120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a178527b84ff446e95c6439e43798348 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:35:37.118161 11754 tablet_bootstrap.cc:654] T 6c42b518ef04429aa6b4e8f5deee05c8 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:37.130607 10924 master-stress-test.cc:580] Killed test thread 2/10
I20260926 07:35:37.130877 10924 master-stress-test.cc:578] Killing test thread 3/10
I20260926 07:35:37.131093 10924 master-stress-test.cc:580] Killed test thread 3/10
I20260926 07:35:37.131268 10924 master-stress-test.cc:578] Killing test thread 4/10
I20260926 07:35:37.135859 11871 ts_tablet_manager.cc:1918] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260926 07:35:37.161166 10924 master-stress-test.cc:580] Killed test thread 4/10
I20260926 07:35:37.161443 10924 master-stress-test.cc:578] Killing test thread 5/10
I20260926 07:35:37.161664 10924 master-stress-test.cc:580] Killed test thread 5/10
I20260926 07:35:37.161873 10924 master-stress-test.cc:578] Killing test thread 6/10
I20260926 07:35:37.162406 10924 master-stress-test.cc:580] Killed test thread 6/10
I20260926 07:35:37.162626 10924 master-stress-test.cc:578] Killing test thread 7/10
I20260926 07:35:37.162918 10924 master-stress-test.cc:580] Killed test thread 7/10
I20260926 07:35:37.163123 10924 master-stress-test.cc:578] Killing test thread 8/10
I20260926 07:35:37.163448 10924 master-stress-test.cc:580] Killed test thread 8/10
I20260926 07:35:37.163637 10924 master-stress-test.cc:578] Killing test thread 9/10
W20260926 07:35:37.104921 11401 consensus_peers.cc:487] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:37.164528 12354 raft_consensus.cc:482] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:37.175050 12354 raft_consensus.cc:504] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
W20260926 07:35:37.175815 11401 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.171.1:41829, user_credentials={real_user=slave}} blocked reactor thread for 70974.6us
I20260926 07:35:37.176802 12354 leader_election.cc:288] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
W20260926 07:35:37.176831 11401 consensus_peers.cc:487] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:37.178450 11315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d524be8509645a89a75b4bac15f0889" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
W20260926 07:35:37.179791 11546 leader_election.cc:341] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:37.180966 11871 log.cc:1196] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/05f071b4db7847a5adbd25a71cac3ea7
I20260926 07:35:37.181737 11871 ts_tablet_manager.cc:1939] T 05f071b4db7847a5adbd25a71cac3ea7 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:37.186318 11754 tablet_bootstrap.cc:492] T 6c42b518ef04429aa6b4e8f5deee05c8 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:37.186759 11754 ts_tablet_manager.cc:1397] T 6c42b518ef04429aa6b4e8f5deee05c8 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.325s user 0.009s sys 0.000s
I20260926 07:35:37.188896 11754 raft_consensus.cc:348] T 6c42b518ef04429aa6b4e8f5deee05c8 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.190085 11754 raft_consensus.cc:374] T 6c42b518ef04429aa6b4e8f5deee05c8 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:37.192533 11756 tablet_bootstrap.cc:654] T 3f581510fe624127a4d27d302f5be614 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:37.193354 11754 raft_consensus.cc:729] T 6c42b518ef04429aa6b4e8f5deee05c8 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:37.194271 11746 tablet_bootstrap.cc:492] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:37.194731 11746 ts_tablet_manager.cc:1397] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.227s user 0.004s sys 0.013s
I20260926 07:35:37.201398 12168 ts_tablet_manager.cc:1918] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:37.201731 12168 log.cc:1196] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/c36c759963cb4915afdca7d12109cc6e
I20260926 07:35:37.202390 12168 ts_tablet_manager.cc:1939] T c36c759963cb4915afdca7d12109cc6e P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:37.207520 11111 master_service.cc:946] ReplaceTablet: received request to replace tablet 52d064b684d0491e91f176d24eddabe1 from {username='slave'} at 127.0.0.1:44836
I20260926 07:35:37.211148 12168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 160) took 3129 ms. Trace:
I20260926 07:35:37.211432 12168 rpcz_store.cc:276] 0926 07:35:34.082013 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.082894 (+ 881us) service_pool.cc:227] Handling call
0926 07:35:36.895049 (+2812155us) ts_tablet_manager.cc:1145] Deleting tablet c36c759963cb4915afdca7d12109cc6e
0926 07:35:36.895095 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:37.027094 (+131999us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:37.201190 (+174096us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:37.211094 (+ 9904us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2805156}
I20260926 07:35:37.212452 11106 catalog_manager.cc:4841] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): tablet c36c759963cb4915afdca7d12109cc6e (table default.Table_b9f92e070aab45cdb8352c1d8a3d603c [id=7245a652e96a445fa9ae35517ac03adf]) successfully deleted
I20260926 07:35:37.212713 12168 tablet_replica.cc:330] T 3e28ca159d8349748812825ac1947c51 P e9bfebb34fa14cd2ad4843fffbe5c54e: stopping tablet replica
I20260926 07:35:37.213387 12168 raft_consensus.cc:2229] T 3e28ca159d8349748812825ac1947c51 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:37.213812 12168 raft_consensus.cc:2258] T 3e28ca159d8349748812825ac1947c51 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:37.215637 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 141) took 5346 ms. Trace:
I20260926 07:35:37.217005 11871 rpcz_store.cc:276] 0926 07:35:31.869122 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:31.869220 (+ 98us) service_pool.cc:227] Handling call
0926 07:35:36.559970 (+4690750us) ts_tablet_manager.cc:1145] Deleting tablet 05f071b4db7847a5adbd25a71cac3ea7
0926 07:35:36.560008 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:36.927818 (+367810us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:37.135645 (+207827us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:37.215578 (+ 79933us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4684901}
I20260926 07:35:37.220386 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 05f071b4db7847a5adbd25a71cac3ea7 (table default.Table_b49978de02e64080bb6633a3b7c7f7ca [id=9e879069d99f41cdb271157af1a3fd1e]) successfully deleted
I20260926 07:35:37.215930 11754 consensus_queue.cc:260] T 6c42b518ef04429aa6b4e8f5deee05c8 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.224797 12168 ts_tablet_manager.cc:1905] T 3e28ca159d8349748812825ac1947c51 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:37.231038 11756 tablet_bootstrap.cc:492] T 3f581510fe624127a4d27d302f5be614 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:37.232462 11871 tablet_replica.cc:330] T a8b629a4aba045b697e33bbebecb05b7 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:37.232995 11756 ts_tablet_manager.cc:1397] T 3f581510fe624127a4d27d302f5be614 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.133s user 0.012s sys 0.004s
I20260926 07:35:37.233209 11871 raft_consensus.cc:2229] T a8b629a4aba045b697e33bbebecb05b7 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:37.233650 11871 raft_consensus.cc:2258] T a8b629a4aba045b697e33bbebecb05b7 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:37.235291 11871 ts_tablet_manager.cc:1905] T a8b629a4aba045b697e33bbebecb05b7 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:37.240638 11754 ts_tablet_manager.cc:1428] T 6c42b518ef04429aa6b4e8f5deee05c8 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.054s user 0.005s sys 0.000s
I20260926 07:35:37.241488 11754 tablet_bootstrap.cc:492] T 21919d22b1d24546911d36384b01d9cc P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:37.242661 11756 raft_consensus.cc:348] T 3f581510fe624127a4d27d302f5be614 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.243372 11756 raft_consensus.cc:374] T 3f581510fe624127a4d27d302f5be614 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:37.243664 11756 raft_consensus.cc:729] T 3f581510fe624127a4d27d302f5be614 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:37.244376 11756 consensus_queue.cc:260] T 3f581510fe624127a4d27d302f5be614 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.262216 11756 ts_tablet_manager.cc:1428] T 3f581510fe624127a4d27d302f5be614 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.028s user 0.006s sys 0.001s
I20260926 07:35:37.268858 11754 tablet_bootstrap.cc:654] T 21919d22b1d24546911d36384b01d9cc P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:37.271477 11756 tablet_bootstrap.cc:492] T c092aff8f2a041ccb09bc4b48e0f359c P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:37.277463 11756 tablet_bootstrap.cc:654] T c092aff8f2a041ccb09bc4b48e0f359c P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:37.253064 12168 ts_tablet_manager.cc:1918] T 3e28ca159d8349748812825ac1947c51 P e9bfebb34fa14cd2ad4843fffbe5c54e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:37.278507 12168 log.cc:1196] T 3e28ca159d8349748812825ac1947c51 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/3e28ca159d8349748812825ac1947c51
I20260926 07:35:37.279253 12168 ts_tablet_manager.cc:1939] T 3e28ca159d8349748812825ac1947c51 P e9bfebb34fa14cd2ad4843fffbe5c54e: Deleting consensus metadata
I20260926 07:35:37.282894 12168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 169) took 2840 ms. Trace:
I20260926 07:35:37.283231 12168 rpcz_store.cc:276] 0926 07:35:34.442385 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.450519 (+ 8134us) service_pool.cc:227] Handling call
0926 07:35:37.212582 (+2762063us) ts_tablet_manager.cc:1145] Deleting tablet 3e28ca159d8349748812825ac1947c51
0926 07:35:37.212621 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:37.247001 (+ 34380us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:37.252894 (+ 5893us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:37.282844 (+ 29950us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2761416}
I20260926 07:35:37.284690 12168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 176) took 2182 ms. Trace:
I20260926 07:35:37.285006 12168 rpcz_store.cc:276] 0926 07:35:35.101990 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:35.102127 (+ 137us) service_pool.cc:227] Handling call
0926 07:35:37.284408 (+2182281us) ts_tablet_manager.cc:1145] Deleting tablet f4aa8bef4bc64953ae2c573238838469
0926 07:35:37.284442 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:37.284645 (+ 203us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2180446}
W20260926 07:35:37.286119 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet f4aa8bef4bc64953ae2c573238838469 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f4aa8bef4bc64953ae2c573238838469 already in progress: creating tablet
I20260926 07:35:37.286412 12168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 178) took 2140 ms. Trace:
I20260926 07:35:37.286674 12168 rpcz_store.cc:276] 0926 07:35:35.146040 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:35.146143 (+ 103us) service_pool.cc:227] Handling call
0926 07:35:37.286145 (+2140002us) ts_tablet_manager.cc:1145] Deleting tablet cef2e44be4084fc78e61cf822659b4ea
0926 07:35:37.286180 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:37.286370 (+ 190us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2129408}
W20260926 07:35:37.288775 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet cef2e44be4084fc78e61cf822659b4ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cef2e44be4084fc78e61cf822659b4ea already in progress: creating tablet
I20260926 07:35:37.289176 12168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 177) took 2182 ms. Trace:
I20260926 07:35:37.290154 12168 rpcz_store.cc:276] 0926 07:35:35.106542 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:35.106782 (+ 240us) service_pool.cc:227] Handling call
0926 07:35:37.288892 (+2182110us) ts_tablet_manager.cc:1145] Deleting tablet c092aff8f2a041ccb09bc4b48e0f359c
0926 07:35:37.288929 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:37.289132 (+ 203us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2112410}
W20260926 07:35:37.291613 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet c092aff8f2a041ccb09bc4b48e0f359c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c092aff8f2a041ccb09bc4b48e0f359c already in progress: creating tablet
I20260926 07:35:37.292577 12168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 180) took 1179 ms. Trace:
I20260926 07:35:37.293223 12168 rpcz_store.cc:276] 0926 07:35:36.112692 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:36.113147 (+ 455us) service_pool.cc:227] Handling call
0926 07:35:37.292266 (+1179119us) ts_tablet_manager.cc:1145] Deleting tablet 0b900e55dcb34eef9f76de31b389e12d
0926 07:35:37.292299 (+ 33us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:37.292511 (+ 212us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1178593}
I20260926 07:35:37.295130 11756 tablet_bootstrap.cc:492] T c092aff8f2a041ccb09bc4b48e0f359c P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
W20260926 07:35:37.295755 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 0b900e55dcb34eef9f76de31b389e12d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0b900e55dcb34eef9f76de31b389e12d already in progress: creating tablet
I20260926 07:35:37.295957 11756 ts_tablet_manager.cc:1397] T c092aff8f2a041ccb09bc4b48e0f359c P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.025s user 0.009s sys 0.003s
I20260926 07:35:37.298147 11756 raft_consensus.cc:348] T c092aff8f2a041ccb09bc4b48e0f359c P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.298807 11756 raft_consensus.cc:374] T c092aff8f2a041ccb09bc4b48e0f359c P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:37.299818 11756 raft_consensus.cc:729] T c092aff8f2a041ccb09bc4b48e0f359c P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:37.300482 11756 consensus_queue.cc:260] T c092aff8f2a041ccb09bc4b48e0f359c P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.299508 12168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52070 (request call id 181) took 1186 ms. Trace:
I20260926 07:35:37.301424 12168 rpcz_store.cc:276] 0926 07:35:36.113023 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:36.113773 (+ 750us) service_pool.cc:227] Handling call
0926 07:35:37.299242 (+1185469us) ts_tablet_manager.cc:1145] Deleting tablet 33ca5d33a5184ce3b4fe88a9a6755124
0926 07:35:37.299275 (+ 33us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:37.299468 (+ 193us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1184959}
W20260926 07:35:37.302412 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 33ca5d33a5184ce3b4fe88a9a6755124 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33ca5d33a5184ce3b4fe88a9a6755124 already in progress: creating tablet
W20260926 07:35:37.303175 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 2dd5175f0ca941f8b3856ea415e2ee68 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2dd5175f0ca941f8b3856ea415e2ee68 already in progress: creating tablet
I20260926 07:35:37.311010 11756 ts_tablet_manager.cc:1428] T c092aff8f2a041ccb09bc4b48e0f359c P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.015s user 0.004s sys 0.000s
I20260926 07:35:37.312569 11756 tablet_bootstrap.cc:492] T 5cd482fb159d418291cb02bc9138b076 P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
W20260926 07:35:37.315250 11106 catalog_manager.cc:4821] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): delete failed for tablet 9be5c479f2544d2dac7da0b311083dcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9be5c479f2544d2dac7da0b311083dcf already in progress: creating tablet
W20260926 07:35:37.315533 11401 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.171.1:41829, user_credentials={real_user=slave}} blocked reactor thread for 137998us
I20260926 07:35:37.317234 11311 raft_consensus.cc:3037] T 248ab84a1f284c2bbddf62f00b485e13 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:37.319437 11754 tablet_bootstrap.cc:492] T 21919d22b1d24546911d36384b01d9cc P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:37.319828 11754 ts_tablet_manager.cc:1397] T 21919d22b1d24546911d36384b01d9cc P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.078s user 0.009s sys 0.000s
I20260926 07:35:37.322337 11754 raft_consensus.cc:348] T 21919d22b1d24546911d36384b01d9cc P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:37.327525 11401 consensus_peers.cc:487] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:37.330780 11756 tablet_bootstrap.cc:654] T 5cd482fb159d418291cb02bc9138b076 P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:37.330508 12388 consensus_queue.cc:1035] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 07:35:37.335614 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet 4239f5d48512403bb711ece401f0756b (table default.Table_686ba040299440d0af09a9fe3501a040 [id=a639ca032a8d4fcdb928635eaa6db0e3]) successfully deleted
I20260926 07:35:37.336237 11754 raft_consensus.cc:374] T 21919d22b1d24546911d36384b01d9cc P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:37.336548 11754 raft_consensus.cc:729] T 21919d22b1d24546911d36384b01d9cc P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:37.337124 11754 consensus_queue.cc:260] T 21919d22b1d24546911d36384b01d9cc P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:37.345613 12259 ts_tablet_manager.cc:1918] T c36c759963cb4915afdca7d12109cc6e P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:37.346057 12259 log.cc:1196] T c36c759963cb4915afdca7d12109cc6e P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/c36c759963cb4915afdca7d12109cc6e
I20260926 07:35:37.346984 12259 ts_tablet_manager.cc:1939] T c36c759963cb4915afdca7d12109cc6e P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:37.348940 11754 ts_tablet_manager.cc:1428] T 21919d22b1d24546911d36384b01d9cc P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.029s user 0.002s sys 0.005s
I20260926 07:35:37.349920 11754 tablet_bootstrap.cc:492] T c1e934f75d604d66aa12c9c8411d137a P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:37.358214 12385 raft_consensus.cc:482] T 3f581510fe624127a4d27d302f5be614 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:37.354447 11871 ts_tablet_manager.cc:1918] T a8b629a4aba045b697e33bbebecb05b7 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:37.358759 11871 log.cc:1196] T a8b629a4aba045b697e33bbebecb05b7 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/a8b629a4aba045b697e33bbebecb05b7
I20260926 07:35:37.358628 12385 raft_consensus.cc:504] T 3f581510fe624127a4d27d302f5be614 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.359579 11871 ts_tablet_manager.cc:1939] T a8b629a4aba045b697e33bbebecb05b7 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:37.360410 12385 leader_election.cc:288] T 3f581510fe624127a4d27d302f5be614 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:37.361063 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f581510fe624127a4d27d302f5be614" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:37.361886 11605 raft_consensus.cc:2379] T 3f581510fe624127a4d27d302f5be614 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9bfebb34fa14cd2ad4843fffbe5c54e in current term 1: Already voted for candidate 22f3902d01584936b20437606cb86dd1 in this term.
I20260926 07:35:37.363924 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 145) took 5365 ms. Trace:
I20260926 07:35:37.366288 11871 rpcz_store.cc:276] 0926 07:35:31.998151 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:31.998265 (+ 114us) service_pool.cc:227] Handling call
0926 07:35:37.232300 (+5234035us) ts_tablet_manager.cc:1145] Deleting tablet a8b629a4aba045b697e33bbebecb05b7
0926 07:35:37.232347 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:37.298366 (+ 66019us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:37.354187 (+ 55821us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:37.363870 (+ 9683us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5233457}
I20260926 07:35:37.368841 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet a8b629a4aba045b697e33bbebecb05b7 (table default.Table_de0d2fdff98146f9aede70ab6a51dbec [id=c469a15ec981495a879b63ec5f961277]) successfully deleted
I20260926 07:35:37.369774 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 146) took 5371 ms. Trace:
I20260926 07:35:37.374683 11871 rpcz_store.cc:276] 0926 07:35:31.998428 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:31.998534 (+ 106us) service_pool.cc:227] Handling call
0926 07:35:37.369560 (+5371026us) ts_tablet_manager.cc:1145] Deleting tablet e06ed286144941269b9a0c0c60bff0ce
0926 07:35:37.369598 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:37.369731 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5370491}
W20260926 07:35:37.375973 11107 catalog_manager.cc:4808] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet e06ed286144941269b9a0c0c60bff0ce because the tablet was not found. No further retry: Not found: Tablet not found: e06ed286144941269b9a0c0c60bff0ce
I20260926 07:35:37.368099 11754 tablet_bootstrap.cc:654] T c1e934f75d604d66aa12c9c8411d137a P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:37.383613 11104 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3e28ca159d8349748812825ac1947c51 (table default.Table_fd842e14c4984402b2b1b77baa1f9360 [id=cc4663f9d1734d9cb4cc4e1f061310a3], current schema version=4) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3e28ca159d8349748812825ac1947c51 consensus state has no leader
I20260926 07:35:37.384044 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 147) took 5384 ms. Trace:
I20260926 07:35:37.385481 11756 tablet_bootstrap.cc:492] T 5cd482fb159d418291cb02bc9138b076 P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:37.385327 11871 rpcz_store.cc:276] 0926 07:35:31.999345 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:31.999439 (+ 94us) service_pool.cc:227] Handling call
0926 07:35:37.379080 (+5379641us) ts_tablet_manager.cc:1145] Deleting tablet 509f607b089c44bc9c3f80ed59a68c95
0926 07:35:37.379121 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:37.383988 (+ 4867us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5379148}
I20260926 07:35:37.385931 11756 ts_tablet_manager.cc:1397] T 5cd482fb159d418291cb02bc9138b076 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.074s user 0.009s sys 0.006s
I20260926 07:35:37.388819 11756 raft_consensus.cc:348] T 5cd482fb159d418291cb02bc9138b076 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.389436 11756 raft_consensus.cc:374] T 5cd482fb159d418291cb02bc9138b076 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260926 07:35:37.389390 11107 catalog_manager.cc:4808] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 509f607b089c44bc9c3f80ed59a68c95 because the tablet was not found. No further retry: Not found: Tablet not found: 509f607b089c44bc9c3f80ed59a68c95
I20260926 07:35:37.389663 11756 raft_consensus.cc:729] T 5cd482fb159d418291cb02bc9138b076 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:37.391139 11756 consensus_queue.cc:260] T 5cd482fb159d418291cb02bc9138b076 P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.390764 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 148) took 5390 ms. Trace:
I20260926 07:35:37.392645 11871 rpcz_store.cc:276] 0926 07:35:32.000218 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:32.000307 (+ 89us) service_pool.cc:227] Handling call
0926 07:35:37.390525 (+5390218us) ts_tablet_manager.cc:1145] Deleting tablet 640d21586bad4070957cff61bd174fd2
0926 07:35:37.390569 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:37.390717 (+ 148us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5389724}
I20260926 07:35:37.397176 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 149) took 5396 ms. Trace:
I20260926 07:35:37.398160 11871 rpcz_store.cc:276] 0926 07:35:32.001085 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:32.001174 (+ 89us) service_pool.cc:227] Handling call
0926 07:35:37.396954 (+5395780us) ts_tablet_manager.cc:1145] Deleting tablet 1620b507f07c446c8c1811895a097409
0926 07:35:37.396996 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:37.397131 (+ 135us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5395321}
I20260926 07:35:37.399539 11756 ts_tablet_manager.cc:1428] T 5cd482fb159d418291cb02bc9138b076 P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
W20260926 07:35:37.409294 11107 catalog_manager.cc:4808] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 1620b507f07c446c8c1811895a097409 because the tablet was not found. No further retry: Not found: Tablet not found: 1620b507f07c446c8c1811895a097409
I20260926 07:35:37.409938 12259 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 162) took 3375 ms. Trace:
I20260926 07:35:37.410784 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 151) took 5392 ms. Trace:
I20260926 07:35:37.410292 12259 rpcz_store.cc:276] 0926 07:35:34.034956 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.035045 (+ 89us) service_pool.cc:227] Handling call
0926 07:35:36.457590 (+2422545us) ts_tablet_manager.cc:1145] Deleting tablet c36c759963cb4915afdca7d12109cc6e
0926 07:35:36.944507 (+486917us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:37.173293 (+228786us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:37.345326 (+172033us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:37.409845 (+ 64519us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":113895,"tablet-delete.queue_time_us":2422047}
I20260926 07:35:37.411609 12259 tablet_replica.cc:330] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:37.412402 12259 raft_consensus.cc:2229] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:37.413165 12259 raft_consensus.cc:2258] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:37.413554 11871 rpcz_store.cc:276] 0926 07:35:32.018229 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:32.018330 (+ 101us) service_pool.cc:227] Handling call
0926 07:35:37.410466 (+5392136us) ts_tablet_manager.cc:1145] Deleting tablet 6d54a5c32da5444ca0a2939461f3a0f6
0926 07:35:37.410510 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:37.410733 (+ 223us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5391615}
I20260926 07:35:37.415140 12259 ts_tablet_manager.cc:1905] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:37.412554 11754 tablet_bootstrap.cc:492] T c1e934f75d604d66aa12c9c8411d137a P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:37.415954 11754 ts_tablet_manager.cc:1397] T c1e934f75d604d66aa12c9c8411d137a P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.066s user 0.009s sys 0.000s
I20260926 07:35:37.417891 11756 tablet_bootstrap.cc:492] T cef2e44be4084fc78e61cf822659b4ea P e9bfebb34fa14cd2ad4843fffbe5c54e: Bootstrap starting.
I20260926 07:35:37.418648 11754 raft_consensus.cc:348] T c1e934f75d604d66aa12c9c8411d137a P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:37.419543 11754 raft_consensus.cc:374] T c1e934f75d604d66aa12c9c8411d137a P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:37.419821 11754 raft_consensus.cc:729] T c1e934f75d604d66aa12c9c8411d137a P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:37.420526 11754 consensus_queue.cc:260] T c1e934f75d604d66aa12c9c8411d137a P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:37.422945 11754 ts_tablet_manager.cc:1428] T c1e934f75d604d66aa12c9c8411d137a P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 07:35:37.423017 11756 tablet_bootstrap.cc:654] T cef2e44be4084fc78e61cf822659b4ea P e9bfebb34fa14cd2ad4843fffbe5c54e: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:37.423725 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 6d54a5c32da5444ca0a2939461f3a0f6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d54a5c32da5444ca0a2939461f3a0f6 already in progress: creating tablet
I20260926 07:35:37.436167 12388 raft_consensus.cc:482] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:37.436795 12388 raft_consensus.cc:504] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.438819 12388 leader_election.cc:288] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:37.440218 11311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8f025d645eb478f95fa0aef17a86729" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" is_pre_election: true
I20260926 07:35:37.454691 12385 raft_consensus.cc:482] T 5cd482fb159d418291cb02bc9138b076 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:37.455117 12385 raft_consensus.cc:504] T 5cd482fb159d418291cb02bc9138b076 P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.457046 12385 leader_election.cc:288] T 5cd482fb159d418291cb02bc9138b076 P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
W20260926 07:35:37.460987 11401 leader_election.cc:341] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:37.461664 11746 raft_consensus.cc:348] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.462754 11746 raft_consensus.cc:374] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:37.463120 11746 raft_consensus.cc:729] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:37.472517 12385 raft_consensus.cc:482] T c092aff8f2a041ccb09bc4b48e0f359c P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:37.472898 12385 raft_consensus.cc:504] T c092aff8f2a041ccb09bc4b48e0f359c P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.474625 12385 leader_election.cc:288] T c092aff8f2a041ccb09bc4b48e0f359c P e9bfebb34fa14cd2ad4843fffbe5c54e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:37.482131 11461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248ab84a1f284c2bbddf62f00b485e13" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:37.493670 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f581510fe624127a4d27d302f5be614" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:37.495160 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cd482fb159d418291cb02bc9138b076" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:37.504696 11756 tablet_bootstrap.cc:492] T cef2e44be4084fc78e61cf822659b4ea P e9bfebb34fa14cd2ad4843fffbe5c54e: No bootstrap required, opened a new log
I20260926 07:35:37.504698 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c092aff8f2a041ccb09bc4b48e0f359c" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:37.505079 11756 ts_tablet_manager.cc:1397] T cef2e44be4084fc78e61cf822659b4ea P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent bootstrapping tablet: real 0.087s user 0.012s sys 0.022s
I20260926 07:35:37.507247 11756 raft_consensus.cc:348] T cef2e44be4084fc78e61cf822659b4ea P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.507903 11756 raft_consensus.cc:374] T cef2e44be4084fc78e61cf822659b4ea P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:37.508280 11756 raft_consensus.cc:729] T cef2e44be4084fc78e61cf822659b4ea P e9bfebb34fa14cd2ad4843fffbe5c54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bfebb34fa14cd2ad4843fffbe5c54e, State: Initialized, Role: FOLLOWER
I20260926 07:35:37.508949 11756 consensus_queue.cc:260] T cef2e44be4084fc78e61cf822659b4ea P e9bfebb34fa14cd2ad4843fffbe5c54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.511174 11756 ts_tablet_manager.cc:1428] T cef2e44be4084fc78e61cf822659b4ea P e9bfebb34fa14cd2ad4843fffbe5c54e: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260926 07:35:37.522416 11444 tablet_service.cc:1467] Processing CreateTablet for tablet 089bac5feeaa4508bc863e446b172033 (DEFAULT_TABLE table=default.Table_1da003049e4b46eb994aaa3f3094d63c [id=3ce63f9fb5ad4b82ba2fbbf2e3678271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.524077 11444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 089bac5feeaa4508bc863e446b172033. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.527398 11111 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: Sending DeleteTablet for 3 replicas of tablet 52d064b684d0491e91f176d24eddabe1
I20260926 07:35:37.529783 11285 tablet_service.cc:1514] Processing DeleteTablet for tablet 52d064b684d0491e91f176d24eddabe1 with delete_type TABLET_DATA_DELETED (replaced by tablet e8d3a481998b4aeda3ef64c000c3aa26) from {username='slave'} at 127.0.0.1:52070
I20260926 07:35:37.532824 11111 catalog_manager.cc:6565] ReplaceTablet: tablet 52d064b684d0491e91f176d24eddabe1 deleted and replaced by tablet e8d3a481998b4aeda3ef64c000c3aa26
I20260926 07:35:37.536442 10924 master-stress-test.cc:580] Killed test thread 9/10
I20260926 07:35:37.536705 10924 master-stress-test.cc:578] Killing test thread 10/10
I20260926 07:35:37.537031 10924 master-stress-test.cc:580] Killed test thread 10/10
I20260926 07:35:37.537340 10924 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu with pid 11202
I20260926 07:35:37.424731 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 157) took 5255 ms. Trace:
I20260926 07:35:37.459575 11607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cd482fb159d418291cb02bc9138b076" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:37.543308 11607 raft_consensus.cc:2379] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9bfebb34fa14cd2ad4843fffbe5c54e in current term 1: Already voted for candidate 22f3902d01584936b20437606cb86dd1 in this term.
I20260926 07:35:37.477231 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c092aff8f2a041ccb09bc4b48e0f359c" candidate_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:37.529803 11581 tablet_service.cc:1514] Processing DeleteTablet for tablet 52d064b684d0491e91f176d24eddabe1 with delete_type TABLET_DATA_DELETED (replaced by tablet e8d3a481998b4aeda3ef64c000c3aa26) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:37.545030 11871 rpcz_store.cc:276] 0926 07:35:32.169074 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:32.169186 (+ 112us) service_pool.cc:227] Handling call
0926 07:35:37.424489 (+5255303us) ts_tablet_manager.cc:1145] Deleting tablet c81a59361a6e42b7962b3e027c4a087f
0926 07:35:37.424532 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:37.424671 (+ 139us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5234032}
W20260926 07:35:37.546775 11107 catalog_manager.cc:4808] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet c81a59361a6e42b7962b3e027c4a087f because the tablet was not found. No further retry: Not found: Tablet not found: c81a59361a6e42b7962b3e027c4a087f
I20260926 07:35:37.547591 11440 tablet_service.cc:1467] Processing CreateTablet for tablet 9be5c479f2544d2dac7da0b311083dcf (DEFAULT_TABLE table=default.Table_597d2bba0c2b4bfc8c1e14e454b61b86 [id=a28d622db9cf420faa75d9aa65e9bcae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.463878 11746 consensus_queue.cc:260] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.548604 11440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9be5c479f2544d2dac7da0b311083dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.549749 11445 tablet_service.cc:1467] Processing CreateTablet for tablet 76eb30d364784ec481e11585bd93446e (DEFAULT_TABLE table=default.Table_7941154c519c4b23a2983c2b3ec82ec2 [id=db4d15f27d9e4c6bb4e0b3f52959b882]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.550572 11445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76eb30d364784ec481e11585bd93446e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.559922 12389 raft_consensus.cc:482] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:37.560324 12389 raft_consensus.cc:504] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.566227 11435 tablet_service.cc:1514] Processing DeleteTablet for tablet c81a59361a6e42b7962b3e027c4a087f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:28 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:37.567178 11435 tablet_service.cc:1514] Processing DeleteTablet for tablet a8b629a4aba045b697e33bbebecb05b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:37.568029 11435 tablet_service.cc:1514] Processing DeleteTablet for tablet 9feaf1ed8ff34928b716fd0b4e41cdde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:29 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:37.568490 12389 leader_election.cc:288] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:37.568574 11115 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 0eae53ce278749208492815b918582f3 (table default.Table_56493b3efdaa43e684afcd3d4b5dffdd [id=f66bffa952604b28bb51299de795ee2a]). Expected version 1 got 0
I20260926 07:35:37.569068 11435 tablet_service.cc:1514] Processing DeleteTablet for tablet 04be706330924282961054c5835cec73 with delete_type TABLET_DATA_DELETED (replaced by tablet 21919d22b1d24546911d36384b01d9cc) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:37.569391 11115 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet a55e54d492214039b0294f954b5ef390 (table default.Table_9158764a5a0d463ebc8413d234878ee7 [id=65c112d56f2a44cc9d840caadd218412]). Expected version 2 got 1
I20260926 07:35:37.570212 11435 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e28ca159d8349748812825ac1947c51 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dd5175f0ca941f8b3856ea415e2ee68) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:37.571074 11435 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e28ca159d8349748812825ac1947c51 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dd5175f0ca941f8b3856ea415e2ee68) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:37.571705 11115 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 248ab84a1f284c2bbddf62f00b485e13 (table default.Table_8439aa0b11574c5b822c46f01f75d842 [id=47d4200172794cc78a6f386fee4b15b4]). Expected version 3 got 0
I20260926 07:35:37.571897 11435 tablet_service.cc:1514] Processing DeleteTablet for tablet 9be5c479f2544d2dac7da0b311083dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:36 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:37.572795 11435 tablet_service.cc:1514] Processing DeleteTablet for tablet 52d064b684d0491e91f176d24eddabe1 with delete_type TABLET_DATA_DELETED (replaced by tablet e8d3a481998b4aeda3ef64c000c3aa26) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:37.574486 11746 ts_tablet_manager.cc:1428] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.379s user 0.006s sys 0.000s
I20260926 07:35:37.575344 12259 ts_tablet_manager.cc:1918] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:37.575749 12259 log.cc:1196] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/04180ace5d4548dfbcd226bf229d5c26
I20260926 07:35:37.576534 12259 ts_tablet_manager.cc:1939] T 04180ace5d4548dfbcd226bf229d5c26 P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:37.579036 11754 tablet_bootstrap.cc:492] T 30df9b4afb414c90af5f4fb0c4582f01 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:37.579428 11871 tablet_replica.cc:330] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:37.580155 11871 raft_consensus.cc:2229] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:37.580552 11871 pending_rounds.cc:62] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade: Trying to abort 1 pending ops.
I20260926 07:35:37.580765 11871 pending_rounds.cc:68] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333511706593038336 op_type: NO_OP noop_request { }
I20260926 07:35:37.581103 11871 raft_consensus.cc:2866] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260926 07:35:37.581403 11871 raft_consensus.cc:2258] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:37.583089 11871 ts_tablet_manager.cc:1905] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:37.587193 11754 tablet_bootstrap.cc:654] T 30df9b4afb414c90af5f4fb0c4582f01 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:37.590642 11581 tablet_service.cc:1467] Processing CreateTablet for tablet 8f9f83078d9e4d57a57b91bd3a70670b (DEFAULT_TABLE table=default.Table_ec15d379852d4e8a8b80c63122bf3bc7 [id=ff00caa43be54a4087d27605b5647ec6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.591681 11581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f9f83078d9e4d57a57b91bd3a70670b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:37.577556 12408 log.cc:924] Time spent T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1: Append to log took a long time: real 0.066s user 0.000s sys 0.000s
I20260926 07:35:37.608453 11444 tablet_service.cc:1467] Processing CreateTablet for tablet 8f9f83078d9e4d57a57b91bd3a70670b (DEFAULT_TABLE table=default.Table_ec15d379852d4e8a8b80c63122bf3bc7 [id=ff00caa43be54a4087d27605b5647ec6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.613423 11583 tablet_service.cc:1467] Processing CreateTablet for tablet 0027166f8f3e408e8d022bb1c39f992d (DEFAULT_TABLE table=default.Table_6fd1abf4390246fa9ab2f95d2c4e76c3 [id=55a30a7c10224e569425698a55b454bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.615769 11445 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dd5175f0ca941f8b3856ea415e2ee68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:35 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:37.616982 11580 tablet_service.cc:1467] Processing CreateTablet for tablet 280e1d605b0245d6b4504af6500ef438 (DEFAULT_TABLE table=default.Table_3748c2123ce44306a4738ddfd5df4847 [id=cb86d75f241e4ae4b9770ad13dab38f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.620391 11586 tablet_service.cc:1467] Processing CreateTablet for tablet 217c3e6f94974492ab8fbe95bac3c421 (DEFAULT_TABLE table=default.Table_286cfdaad687460f8aa8dff237604241 [id=8907065e82594f7d9b87ab007616c353]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.616528 12389 consensus_queue.cc:1035] T 3f581510fe624127a4d27d302f5be614 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:37.617478 11746 tablet_bootstrap.cc:492] T 6d524be8509645a89a75b4bac15f0889 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:37.631100 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0027166f8f3e408e8d022bb1c39f992d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.636610 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 280e1d605b0245d6b4504af6500ef438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.637092 11586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 217c3e6f94974492ab8fbe95bac3c421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.640872 11120 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet a55e54d492214039b0294f954b5ef390 (table default.Table_9158764a5a0d463ebc8413d234878ee7 [id=65c112d56f2a44cc9d840caadd218412]). Expected version 2 got 1
I20260926 07:35:37.643765 11120 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 0eae53ce278749208492815b918582f3 (table default.Table_56493b3efdaa43e684afcd3d4b5dffdd [id=f66bffa952604b28bb51299de795ee2a]). Expected version 1 got 0
I20260926 07:35:37.652876 11120 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 248ab84a1f284c2bbddf62f00b485e13 (table default.Table_8439aa0b11574c5b822c46f01f75d842 [id=47d4200172794cc78a6f386fee4b15b4]). Expected version 3 got 0
I20260926 07:35:37.660058 12388 consensus_queue.cc:1035] T 5cd482fb159d418291cb02bc9138b076 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:37.663240 12388 consensus_queue.cc:1035] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:37.619159 12259 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 163) took 3583 ms. Trace:
I20260926 07:35:37.664794 12259 rpcz_store.cc:276] 0926 07:35:34.035389 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.035623 (+ 234us) service_pool.cc:227] Handling call
0926 07:35:37.411441 (+3375818us) ts_tablet_manager.cc:1145] Deleting tablet 04180ace5d4548dfbcd226bf229d5c26
0926 07:35:37.411507 (+ 66us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:37.464412 (+ 52905us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:37.575133 (+110721us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:37.619105 (+ 43972us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1742,"tablet-delete.queue_time_us":3375294}
I20260926 07:35:37.671970 11871 ts_tablet_manager.cc:1918] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:37.673151 11118 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: default.Table_e8b00b73be7c4818adf8d1e74d731ee6 [id=ddb5eb3862f54c0cb6f3c1cf311f4a9b] alter complete (version 3)
I20260926 07:35:37.674463 12259 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 164) took 3637 ms. Trace:
I20260926 07:35:37.676972 11871 log.cc:1196] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/55ad1d96e2314d1eb0a717a3e9ff31bc
I20260926 07:35:37.679893 11871 ts_tablet_manager.cc:1939] T 55ad1d96e2314d1eb0a717a3e9ff31bc P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:37.665874 12388 consensus_queue.cc:1035] T 52d064b684d0491e91f176d24eddabe1 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:37.672667 12389 consensus_queue.cc:1035] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:37.620563 11444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f9f83078d9e4d57a57b91bd3a70670b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.686120 12259 rpcz_store.cc:276] 0926 07:35:34.037471 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.037568 (+ 97us) service_pool.cc:227] Handling call
0926 07:35:37.674171 (+3636603us) ts_tablet_manager.cc:1145] Deleting tablet 034d508f0a164f3ea8308ea2d28818c4
0926 07:35:37.674229 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:37.674407 (+ 178us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3636106}
W20260926 07:35:37.686538 11112 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.1:34359 (request call id 47) took 2165 ms (client timeout 1000 ms). Trace:
W20260926 07:35:37.686837 11112 rpcz_store.cc:269] 0926 07:35:35.520814 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:35.520939 (+ 125us) service_pool.cc:227] Handling call
0926 07:35:37.686475 (+2165536us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6897,"reported_tablets":15}
W20260926 07:35:37.688696 11113 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.1:34359 (request call id 48) took 1147 ms (client timeout 1000 ms). Trace:
I20260926 07:35:37.683748 11580 tablet_service.cc:1467] Processing CreateTablet for tablet b9725ef33479457eae7b987fda0d9949 (DEFAULT_TABLE table=default.Table_87b25251764145f7b16c7d779094b2a2 [id=6e78640dac1a4a7f85d3a07cc4748a66]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 07:35:37.689021 11113 rpcz_store.cc:269] 0926 07:35:36.541137 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:36.547359 (+ 6222us) service_pool.cc:227] Handling call
0926 07:35:37.688639 (+1141280us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":984,"reported_tablets":16}
W20260926 07:35:37.690128 11119 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.1:34359 (request call id 46) took 3545 ms (client timeout 1000 ms). Trace:
I20260926 07:35:37.690335 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9725ef33479457eae7b987fda0d9949. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:37.690428 11119 rpcz_store.cc:269] 0926 07:35:34.144720 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.173324 (+ 28604us) service_pool.cc:227] Handling call
0926 07:35:34.629660 (+456336us) spinlock_profiling.cc:243] Waited 73 ms on lock 0x7b54000118d4. stack: 00007fb80941f033 00007fb80941ef30 00007fb808fa0534 00000000004db2b4 00000000004db249 00007fb81032baf0 00007fb8103aef2a 00007fb81227ad6c 00007fb812374c3e 00007fb80feabfb4 00007fb80feabf42 00007fb80feabe71 00007fb80feabdec 00007fb80feab0a2 00007fb80e15594c 00007fb80e154d86
0926 07:35:37.690062 (+3060402us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:35.554998 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:35.555355 (+ 357us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:35.555380 (+ 25us) write_op.cc:435] Acquired schema lock
0926 07:35:35.555394 (+ 14us) tablet.cc:660] Decoding operations
0926 07:35:35.555633 (+ 239us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:35.555673 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:35.555718 (+ 45us) tablet.cc:683] Acquiring locks for 1 operations
0926 07:35:35.555856 (+ 138us) tablet.cc:699] Row locks acquired
0926 07:35:35.555870 (+ 14us) write_op.cc:260] PREPARE: finished
0926 07:35:35.555966 (+ 96us) write_op.cc:270] Start()
0926 07:35:35.556039 (+ 73us) write_op.cc:276] Timestamp: P: 1790408135555912 usec, L: 0
0926 07:35:35.556066 (+ 27us) op_driver.cc:331] REPLICATION: starting
0926 07:35:35.558371 (+ 2305us) log.cc:842] Serialized 356 byte log entry
0926 07:35:35.611440 (+ 53069us) op_driver.cc:444] REPLICATION: finished
0926 07:35:35.614364 (+ 2924us) write_op.cc:301] APPLY: starting
0926 07:35:35.614418 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:35.614501 (+ 83us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:35.614602 (+ 101us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:35.614749 (+ 147us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:35.614871 (+ 122us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:35.614917 (+ 46us) write_op.cc:312] APPLY: finished
0926 07:35:35.615080 (+ 163us) log.cc:842] Serialized 23 byte log entry
0926 07:35:35.615218 (+ 138us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:35.615280 (+ 62us) write_op.cc:454] Released schema lock
0926 07:35:35.615326 (+ 46us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:35.615481 (+ 155us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:35.615503 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6804,"reported_tablets":15,"spinlock_wait_cycles":167990144,"child_traces":[["op",{"apply.queue_time_us":2768,"apply.run_cpu_time_us":1295,"apply.run_wall_time_us":1294,"num_ops":1,"prepare.queue_time_us":2123,"prepare.run_cpu_time_us":1799,"prepare.run_wall_time_us":8256,"raft.queue_time_us":920,"raft.run_cpu_time_us":538,"raft.run_wall_time_us":538,"replication_time_us":55286}]]}
I20260926 07:35:37.695652 11581 tablet_service.cc:1467] Processing CreateTablet for tablet 85f987ee219941cb824e0d68b5660b55 (DEFAULT_TABLE table=default.Table_1330df49c5964507ba4c568038f33499 [id=73d12ba7ff404bdf9ad19c710094c872]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 07:35:37.696480 11118 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.2:33317 (request call id 51) took 1551 ms (client timeout 1000 ms). Trace:
I20260926 07:35:37.696624 11581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85f987ee219941cb824e0d68b5660b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.696595 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25b8b701563344bbbc986db2b0f39b9b" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:37.698262 12259 tablet_replica.cc:330] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:37.706604 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d524be8509645a89a75b4bac15f0889" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:37.709425 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet c36c759963cb4915afdca7d12109cc6e (table default.Table_b9f92e070aab45cdb8352c1d8a3d603c [id=7245a652e96a445fa9ae35517ac03adf]) successfully deleted
I20260926 07:35:37.710120 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8f025d645eb478f95fa0aef17a86729" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:37.710783 11605 raft_consensus.cc:2454] T f8f025d645eb478f95fa0aef17a86729 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:37.712316 11583 tablet_service.cc:1467] Processing CreateTablet for tablet 97508af7402a4d1dae78d8b0d66b01b4 (DEFAULT_TABLE table=default.Table_405e87138f64473890e7ba5b74d483eb [id=e07c336170534e52b2993ed07016649e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.713323 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97508af7402a4d1dae78d8b0d66b01b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.717511 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 155) took 5549 ms. Trace:
I20260926 07:35:37.719374 11746 tablet_bootstrap.cc:654] T 6d524be8509645a89a75b4bac15f0889 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:37.696846 11118 rpcz_store.cc:269] 0926 07:35:36.144724 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:36.287316 (+142592us) service_pool.cc:227] Handling call
0926 07:35:37.041266 (+753950us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:37.041291 (+ 25us) write_op.cc:454] Released schema lock
0926 07:35:37.696424 (+655133us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 07:35:36.768322 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:36.768670 (+ 348us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:36.768694 (+ 24us) write_op.cc:435] Acquired schema lock
0926 07:35:36.768707 (+ 13us) tablet.cc:660] Decoding operations
0926 07:35:36.769157 (+ 450us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:36.769198 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:36.769220 (+ 22us) tablet.cc:683] Acquiring locks for 4 operations
0926 07:35:36.769497 (+ 277us) tablet.cc:699] Row locks acquired
0926 07:35:36.769510 (+ 13us) write_op.cc:260] PREPARE: finished
0926 07:35:36.769625 (+ 115us) write_op.cc:270] Start()
0926 07:35:36.769704 (+ 79us) write_op.cc:276] Timestamp: P: 1790408136769582 usec, L: 0
0926 07:35:36.769719 (+ 15us) op_driver.cc:331] REPLICATION: starting
0926 07:35:36.770181 (+ 462us) log.cc:842] Serialized 1662 byte log entry
0926 07:35:36.777728 (+ 7547us) op_driver.cc:444] REPLICATION: finished
0926 07:35:36.778035 (+ 307us) write_op.cc:301] APPLY: starting
0926 07:35:36.778087 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:36.778211 (+ 124us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:36.778307 (+ 96us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:36.778771 (+ 464us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:36.778929 (+ 158us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0926 07:35:36.778971 (+ 42us) write_op.cc:312] APPLY: finished
0926 07:35:36.779178 (+ 207us) log.cc:842] Serialized 41 byte log entry
0926 07:35:36.779304 (+ 126us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:36.779382 (+ 78us) write_op.cc:454] Released schema lock
0926 07:35:36.779413 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0926 07:35:36.779574 (+ 161us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:36.779596 (+ 22us) write_op.cc:454] Released schema lock
Related trace 'op':
0926 07:35:36.936322 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 07:35:36.936680 (+ 358us) write_op.cc:432] Acquiring schema lock in shared mode
0926 07:35:36.936705 (+ 25us) write_op.cc:435] Acquired schema lock
0926 07:35:36.936720 (+ 15us) tablet.cc:660] Decoding operations
0926 07:35:36.936964 (+ 244us) write_op.cc:620] Acquiring the partition lock for write op
0926 07:35:36.937006 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0926 07:35:36.937027 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0926 07:35:36.937162 (+ 135us) tablet.cc:699] Row locks acquired
0926 07:35:36.937177 (+ 15us) write_op.cc:260] PREPARE: finished
0926 07:35:36.937295 (+ 118us) write_op.cc:270] Start()
0926 07:35:36.937383 (+ 88us) write_op.cc:276] Timestamp: P: 1790408136937221 usec, L: 0
0926 07:35:36.937399 (+ 16us) op_driver.cc:331] REPLICATION: starting
0926 07:35:36.937879 (+ 480us) log.cc:842] Serialized 356 byte log entry
0926 07:35:36.955038 (+ 17159us) op_driver.cc:444] REPLICATION: finished
0926 07:35:36.955301 (+ 263us) write_op.cc:301] APPLY: starting
0926 07:35:36.955353 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0926 07:35:36.955443 (+ 90us) tablet.cc:1368] finished BulkCheckPresence
0926 07:35:36.955530 (+ 87us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 07:35:36.955661 (+ 131us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 07:35:36.955794 (+ 133us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 07:35:36.955833 (+ 39us) write_op.cc:312] APPLY: finished
0926 07:35:36.955981 (+ 148us) log.cc:842] Serialized 23 byte log entry
0926 07:35:36.956101 (+ 120us) write_op.cc:489] Releasing partition, row and schema locks
0926 07:35:36.956181 (+ 80us) write_op.cc:454] Released schema lock
0926 07:35:36.956209 (+ 28us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":68131,"reported_tablets":23,"child_traces":[["op",{"apply.queue_time_us":173,"apply.run_cpu_time_us":1737,"apply.run_wall_time_us":1736,"mutex_wait_us":1496,"num_ops":4,"prepare.queue_time_us":45021,"prepare.run_cpu_time_us":2222,"prepare.run_wall_time_us":3700,"raft.queue_time_us":1751,"raft.run_cpu_time_us":1343,"raft.run_wall_time_us":1343,"replication_time_us":7934}],["op",{"apply.queue_time_us":142,"apply.run_cpu_time_us":1085,"apply.run_wall_time_us":1085,"num_ops":1,"prepare.queue_time_us":80931,"prepare.run_cpu_time_us":1862,"prepare.run_wall_time_us":1862,"raft.queue_time_us":3356,"raft.run_cpu_time_us":1375,"raft.run_wall_time_us":1375,"replication_time_us":17572}]]}
I20260926 07:35:37.715926 12259 raft_consensus.cc:2229] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:37.720041 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e926ee35854c28a8c698384526fe38" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:37.721171 11605 raft_consensus.cc:2454] T b8e926ee35854c28a8c698384526fe38 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:37.722551 12259 raft_consensus.cc:2258] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:37.722909 11871 rpcz_store.cc:276] 0926 07:35:32.168483 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:32.169209 (+ 726us) service_pool.cc:227] Handling call
0926 07:35:37.547640 (+5378431us) ts_tablet_manager.cc:1145] Deleting tablet 55ad1d96e2314d1eb0a717a3e9ff31bc
0926 07:35:37.579312 (+ 31672us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:37.606259 (+ 26947us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:37.671758 (+ 65499us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:37.717461 (+ 45703us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5356316}
I20260926 07:35:37.726541 11754 tablet_bootstrap.cc:492] T 30df9b4afb414c90af5f4fb0c4582f01 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:37.726967 11754 ts_tablet_manager.cc:1397] T 30df9b4afb414c90af5f4fb0c4582f01 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.148s user 0.008s sys 0.009s
I20260926 07:35:37.728585 11871 tablet_replica.cc:330] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:37.725095 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 55ad1d96e2314d1eb0a717a3e9ff31bc (table default.Table_4d463023afd047528c16b7ee4dabd36f [id=eeb221378f35425eb8d24b815f8a2181]) successfully deleted
I20260926 07:35:37.729212 11754 raft_consensus.cc:348] T 30df9b4afb414c90af5f4fb0c4582f01 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:37.738651 11590 tablet_service.cc:1467] Processing CreateTablet for tablet 45eb1b12815847d3ae91a05d6b0db4f3 (DEFAULT_TABLE table=default.Table_a3b47c514b814ebba363e6a1fa5c1743 [id=fb6c083946de41c1bda618cb605ea8a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.768271 11591 tablet_service.cc:1467] Processing CreateTablet for tablet cff1a241497f434bb282184a18bbe6e3 (DEFAULT_TABLE table=default.Table_665b626e1a0545cfbbb36ef06105dc06 [id=01468772283248b391e83e5838782e3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.768714 11590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45eb1b12815847d3ae91a05d6b0db4f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.769227 11591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cff1a241497f434bb282184a18bbe6e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.741148 11586 tablet_service.cc:1467] Processing CreateTablet for tablet 361ab96150254adaa58eb02e1dba0300 (DEFAULT_TABLE table=default.Table_c3edf5d221b54d5b86c24973e57cec68 [id=198ebb9d135247f2aef4698d5ce972fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.772737 11586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 361ab96150254adaa58eb02e1dba0300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.749974 11581 tablet_service.cc:1467] Processing CreateTablet for tablet 46892650298e4262888371a09843fbda (DEFAULT_TABLE table=default.Table_e7e2b6923f804b9fac3a16fed6570765 [id=687bb93e41694e3dbb6840412a7df19f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.776636 11581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46892650298e4262888371a09843fbda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.776597 11754 raft_consensus.cc:374] T 30df9b4afb414c90af5f4fb0c4582f01 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:37.776929 11754 raft_consensus.cc:729] T 30df9b4afb414c90af5f4fb0c4582f01 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:37.777323 12259 ts_tablet_manager.cc:1905] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:37.758023 11588 tablet_service.cc:1467] Processing CreateTablet for tablet 3e1ea29b71c64497bb427c575b78fe01 (DEFAULT_TABLE table=default.Table_81f434ad83d24e2b9abc65b59e28381e [id=965045547daf4502814c7296026c3187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.780725 11588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e1ea29b71c64497bb427c575b78fe01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.781025 11871 raft_consensus.cc:2229] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:37.781435 11871 raft_consensus.cc:2258] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:37.785296 11871 ts_tablet_manager.cc:1905] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:37.796002 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet 04180ace5d4548dfbcd226bf229d5c26 (table default.Table_2f42e2e9196745bb942de35a1373e040 [id=abb24fc02fd44eb98163e5c7ad16cf63]) successfully deleted
W20260926 07:35:37.838328 11105 catalog_manager.cc:4808] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 034d508f0a164f3ea8308ea2d28818c4 because the tablet was not found. No further retry: Not found: Tablet not found: 034d508f0a164f3ea8308ea2d28818c4
I20260926 07:35:37.839514 11440 tablet_service.cc:1467] Processing CreateTablet for tablet 0027166f8f3e408e8d022bb1c39f992d (DEFAULT_TABLE table=default.Table_6fd1abf4390246fa9ab2f95d2c4e76c3 [id=55a30a7c10224e569425698a55b454bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.840550 11440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0027166f8f3e408e8d022bb1c39f992d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.740099 11582 tablet_service.cc:1467] Processing CreateTablet for tablet 0ab67c468d0d4e5cb89db391ec1fed65 (DEFAULT_TABLE table=default.Table_ed9c48cda45b4d4fbf739fbea19e7943 [id=9973dfaca8174d5dbab9b5ed866a8b5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.760531 11580 tablet_service.cc:1467] Processing CreateTablet for tablet fb6e0beebafc493daec9370e2ecce90c (DEFAULT_TABLE table=default.Table_fff130a3d2e2461ba2e11f11de45bdf4 [id=3621019745a94fee8bb96a45ee124b5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.796337 11754 consensus_queue.cc:260] T 30df9b4afb414c90af5f4fb0c4582f01 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:37.816846 12133 raft_consensus.cc:482] T 6c42b518ef04429aa6b4e8f5deee05c8 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:37.844848 11582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ab67c468d0d4e5cb89db391ec1fed65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.844689 12133 raft_consensus.cc:504] T 6c42b518ef04429aa6b4e8f5deee05c8 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.847402 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb6e0beebafc493daec9370e2ecce90c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.825371 12354 raft_consensus.cc:482] T 21919d22b1d24546911d36384b01d9cc P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:37.851539 12354 raft_consensus.cc:504] T 21919d22b1d24546911d36384b01d9cc P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:37.853420 12354 leader_election.cc:288] T 21919d22b1d24546911d36384b01d9cc P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:37.826051 11589 tablet_service.cc:1467] Processing CreateTablet for tablet ecd4c35671e54b2d91e720017079224f (DEFAULT_TABLE table=default.Table_a4c0b35e65b84427bc8f867d5717dc3a [id=c7168a14bec5426faab1cbe1f788612d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.826730 11590 tablet_service.cc:1514] Processing DeleteTablet for tablet 05f071b4db7847a5adbd25a71cac3ea7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:30 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:37.855453 11589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd4c35671e54b2d91e720017079224f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.828637 11588 tablet_service.cc:1514] Processing DeleteTablet for tablet 55ad1d96e2314d1eb0a717a3e9ff31bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:37.860793 12133 leader_election.cc:288] T 6c42b518ef04429aa6b4e8f5deee05c8 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:37.862066 11746 tablet_bootstrap.cc:492] T 6d524be8509645a89a75b4bac15f0889 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:37.862462 11746 ts_tablet_manager.cc:1397] T 6d524be8509645a89a75b4bac15f0889 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.245s user 0.019s sys 0.002s
I20260926 07:35:37.864938 11746 raft_consensus.cc:348] T 6d524be8509645a89a75b4bac15f0889 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.865731 11746 raft_consensus.cc:374] T 6d524be8509645a89a75b4bac15f0889 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:37.865942 11746 raft_consensus.cc:729] T 6d524be8509645a89a75b4bac15f0889 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:37.866525 11746 consensus_queue.cc:260] T 6d524be8509645a89a75b4bac15f0889 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:37.869639 11445 tablet_service.cc:1467] Processing CreateTablet for tablet 280e1d605b0245d6b4504af6500ef438 (DEFAULT_TABLE table=default.Table_3748c2123ce44306a4738ddfd5df4847 [id=cb86d75f241e4ae4b9770ad13dab38f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.869982 11754 ts_tablet_manager.cc:1428] T 30df9b4afb414c90af5f4fb0c4582f01 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.143s user 0.005s sys 0.000s
I20260926 07:35:37.871134 11435 tablet_service.cc:1467] Processing CreateTablet for tablet 217c3e6f94974492ab8fbe95bac3c421 (DEFAULT_TABLE table=default.Table_286cfdaad687460f8aa8dff237604241 [id=8907065e82594f7d9b87ab007616c353]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.872794 11445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 280e1d605b0245d6b4504af6500ef438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.873696 11437 tablet_service.cc:1467] Processing CreateTablet for tablet 85f987ee219941cb824e0d68b5660b55 (DEFAULT_TABLE table=default.Table_1330df49c5964507ba4c568038f33499 [id=73d12ba7ff404bdf9ad19c710094c872]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.875241 11754 tablet_bootstrap.cc:492] T 5ee33141350b4c31aae79d2160e79b6f P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:37.875386 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet 4239f5d48512403bb711ece401f0756b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:28 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:37.876798 11442 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cd482fb159d418291cb02bc9138b076 with delete_type TABLET_DATA_DELETED (replaced by tablet 393da59b3671473ca9714d3406dd1388) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:37.876996 11439 tablet_service.cc:1467] Processing CreateTablet for tablet b9725ef33479457eae7b987fda0d9949 (DEFAULT_TABLE table=default.Table_87b25251764145f7b16c7d779094b2a2 [id=6e78640dac1a4a7f85d3a07cc4748a66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.878593 11442 tablet_service.cc:1467] Processing CreateTablet for tablet 97508af7402a4d1dae78d8b0d66b01b4 (DEFAULT_TABLE table=default.Table_405e87138f64473890e7ba5b74d483eb [id=e07c336170534e52b2993ed07016649e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.892726 11754 tablet_bootstrap.cc:654] T 5ee33141350b4c31aae79d2160e79b6f P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:37.892913 11437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85f987ee219941cb824e0d68b5660b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.900385 11439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9725ef33479457eae7b987fda0d9949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.904577 11442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97508af7402a4d1dae78d8b0d66b01b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.916561 11435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 217c3e6f94974492ab8fbe95bac3c421. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:37.928246 11545 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.171.2:42379, user_credentials={real_user=slave}} blocked reactor thread for 100737us
W20260926 07:35:37.929956 11545 leader_election.cc:341] T 25b8b701563344bbbc986db2b0f39b9b P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:37.930303 11545 leader_election.cc:302] T 25b8b701563344bbbc986db2b0f39b9b P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:37.931809 12354 raft_consensus.cc:2726] T 25b8b701563344bbbc986db2b0f39b9b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:35:37.932190 11545 leader_election.cc:341] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:37.932533 11545 leader_election.cc:302] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:37.934177 11746 ts_tablet_manager.cc:1428] T 6d524be8509645a89a75b4bac15f0889 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.071s user 0.005s sys 0.000s
W20260926 07:35:37.942817 11115 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.171.3:53335 (request call id 34) took 1190 ms (client timeout 1000 ms). Trace:
W20260926 07:35:37.943137 11115 rpcz_store.cc:269] 0926 07:35:36.751981 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:36.826349 (+ 74368us) service_pool.cc:227] Handling call
0926 07:35:37.942753 (+1116404us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":193,"reported_tablets":29}
I20260926 07:35:37.949693 11754 tablet_bootstrap.cc:492] T 5ee33141350b4c31aae79d2160e79b6f P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:37.956432 11746 tablet_bootstrap.cc:492] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:37.961109 11754 ts_tablet_manager.cc:1397] T 5ee33141350b4c31aae79d2160e79b6f P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.075s user 0.007s sys 0.020s
I20260926 07:35:37.964732 11754 raft_consensus.cc:348] T 5ee33141350b4c31aae79d2160e79b6f P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:37.965365 11754 raft_consensus.cc:374] T 5ee33141350b4c31aae79d2160e79b6f P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:37.965601 11754 raft_consensus.cc:729] T 5ee33141350b4c31aae79d2160e79b6f P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:37.966120 11754 consensus_queue.cc:260] T 5ee33141350b4c31aae79d2160e79b6f P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:37.976434 11754 ts_tablet_manager.cc:1428] T 5ee33141350b4c31aae79d2160e79b6f P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.015s user 0.001s sys 0.004s
I20260926 07:35:37.977186 11754 tablet_bootstrap.cc:492] T e5d942ed0a7b4f81bb9bfb915b5512f7 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:37.977727 11746 tablet_bootstrap.cc:654] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:37.981702 11754 tablet_bootstrap.cc:654] T e5d942ed0a7b4f81bb9bfb915b5512f7 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:37.990536 11445 tablet_service.cc:1467] Processing CreateTablet for tablet 0ab67c468d0d4e5cb89db391ec1fed65 (DEFAULT_TABLE table=default.Table_ed9c48cda45b4d4fbf739fbea19e7943 [id=9973dfaca8174d5dbab9b5ed866a8b5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.990967 11440 tablet_service.cc:1467] Processing CreateTablet for tablet fb6e0beebafc493daec9370e2ecce90c (DEFAULT_TABLE table=default.Table_fff130a3d2e2461ba2e11f11de45bdf4 [id=3621019745a94fee8bb96a45ee124b5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.992558 11440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb6e0beebafc493daec9370e2ecce90c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.993252 11446 tablet_service.cc:1467] Processing CreateTablet for tablet cff1a241497f434bb282184a18bbe6e3 (DEFAULT_TABLE table=default.Table_665b626e1a0545cfbbb36ef06105dc06 [id=01468772283248b391e83e5838782e3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.994386 11446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cff1a241497f434bb282184a18bbe6e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.996835 11436 tablet_service.cc:1467] Processing CreateTablet for tablet 3e1ea29b71c64497bb427c575b78fe01 (DEFAULT_TABLE table=default.Table_81f434ad83d24e2b9abc65b59e28381e [id=965045547daf4502814c7296026c3187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:37.997766 11436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e1ea29b71c64497bb427c575b78fe01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.998450 11118 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 0eae53ce278749208492815b918582f3 (table default.Table_56493b3efdaa43e684afcd3d4b5dffdd [id=f66bffa952604b28bb51299de795ee2a]). Expected version 1 got 0
I20260926 07:35:38.001121 12388 raft_consensus.cc:482] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:38.001353 11443 tablet_service.cc:1467] Processing CreateTablet for tablet ecd4c35671e54b2d91e720017079224f (DEFAULT_TABLE table=default.Table_a4c0b35e65b84427bc8f867d5717dc3a [id=c7168a14bec5426faab1cbe1f788612d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:38.001528 12388 raft_consensus.cc:504] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:38.002350 11443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd4c35671e54b2d91e720017079224f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:38.004055 12388 leader_election.cc:288] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:38.006320 11444 tablet_service.cc:1467] Processing CreateTablet for tablet 45eb1b12815847d3ae91a05d6b0db4f3 (DEFAULT_TABLE table=default.Table_a3b47c514b814ebba363e6a1fa5c1743 [id=fb6c083946de41c1bda618cb605ea8a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:38.007386 11444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45eb1b12815847d3ae91a05d6b0db4f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:37.991580 11435 tablet_service.cc:1467] Processing CreateTablet for tablet 46892650298e4262888371a09843fbda (DEFAULT_TABLE table=default.Table_e7e2b6923f804b9fac3a16fed6570765 [id=687bb93e41694e3dbb6840412a7df19f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:38.008762 11435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46892650298e4262888371a09843fbda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:38.010394 11445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ab67c468d0d4e5cb89db391ec1fed65. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 07:35:38.015491 11545 raft_consensus.cc:2670] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260926 07:35:38.016036 11545 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.2:42379, user_credentials={real_user=slave}} blocked reactor thread for 83866.2us
I20260926 07:35:38.016350 11754 tablet_bootstrap.cc:492] T e5d942ed0a7b4f81bb9bfb915b5512f7 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
W20260926 07:35:38.016782 11545 consensus_peers.cc:487] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade -> Peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Couldn't send request to peer 22f3902d01584936b20437606cb86dd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c36c759963cb4915afdca7d12109cc6e. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:38.016862 11754 ts_tablet_manager.cc:1397] T e5d942ed0a7b4f81bb9bfb915b5512f7 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.040s user 0.008s sys 0.000s
I20260926 07:35:38.019122 11754 raft_consensus.cc:348] T e5d942ed0a7b4f81bb9bfb915b5512f7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.019729 11754 raft_consensus.cc:374] T e5d942ed0a7b4f81bb9bfb915b5512f7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:38.019958 11754 raft_consensus.cc:729] T e5d942ed0a7b4f81bb9bfb915b5512f7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:38.020592 11754 consensus_queue.cc:260] T e5d942ed0a7b4f81bb9bfb915b5512f7 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.028730 11754 ts_tablet_manager.cc:1428] T e5d942ed0a7b4f81bb9bfb915b5512f7 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260926 07:35:38.030153 11438 tablet_service.cc:1514] Processing DeleteTablet for tablet c36c759963cb4915afdca7d12109cc6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:27 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:38.032024 11438 tablet_service.cc:1514] Processing DeleteTablet for tablet 04180ace5d4548dfbcd226bf229d5c26 with delete_type TABLET_DATA_DELETED (replaced by tablet b862f325a9df49079066dc62fe1a61f1) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:38.036708 11754 tablet_bootstrap.cc:492] T 0b1b14a340a7474fa8e76a464b0838e1 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:38.039702 11580 tablet_service.cc:1514] Processing DeleteTablet for tablet a8b629a4aba045b697e33bbebecb05b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:25 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:38.055258 11118 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 248ab84a1f284c2bbddf62f00b485e13 (table default.Table_8439aa0b11574c5b822c46f01f75d842 [id=47d4200172794cc78a6f386fee4b15b4]). Expected version 3 got 0
I20260926 07:35:38.056376 11118 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 6c42b518ef04429aa6b4e8f5deee05c8 (table default.Table_3de17a801f1c4fa99878e07e9df543f8 [id=0ebbd3b8ecd242c2a21e0630227a01c7]). Expected version 2 got 0
I20260926 07:35:38.057749 11754 tablet_bootstrap.cc:654] T 0b1b14a340a7474fa8e76a464b0838e1 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:38.065815 11439 tablet_service.cc:1467] Processing CreateTablet for tablet 361ab96150254adaa58eb02e1dba0300 (DEFAULT_TABLE table=default.Table_c3edf5d221b54d5b86c24973e57cec68 [id=198ebb9d135247f2aef4698d5ce972fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:38.066804 11439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 361ab96150254adaa58eb02e1dba0300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:38.070119 11607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9f7faf55684e3e8512a033aa1443fe" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:38.070215 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c42b518ef04429aa6b4e8f5deee05c8" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:38.070662 11607 raft_consensus.cc:2454] T ed9f7faf55684e3e8512a033aa1443fe P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:38.071655 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21919d22b1d24546911d36384b01d9cc" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:38.073491 11402 leader_election.cc:302] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters: e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:38.074193 12388 raft_consensus.cc:2781] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:38.074497 12388 raft_consensus.cc:482] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:38.074756 12388 raft_consensus.cc:3037] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:38.087288 11746 tablet_bootstrap.cc:492] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:38.087704 11746 ts_tablet_manager.cc:1397] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.131s user 0.008s sys 0.000s
I20260926 07:35:38.083967 12388 raft_consensus.cc:504] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:38.089856 11746 raft_consensus.cc:348] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:38.090462 11746 raft_consensus.cc:374] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:38.090773 11746 raft_consensus.cc:729] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:38.091002 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet 25b8b701563344bbbc986db2b0f39b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:29 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:38.091390 11746 consensus_queue.cc:260] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:38.108270 12388 leader_election.cc:288] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:38.110770 12389 raft_consensus.cc:482] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:38.111167 12389 raft_consensus.cc:504] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:38.112866 12389 leader_election.cc:288] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:38.118209 11746 ts_tablet_manager.cc:1428] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.030s user 0.005s sys 0.001s
I20260926 07:35:38.123937 11746 tablet_bootstrap.cc:492] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
W20260926 07:35:38.120502 11402 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.3:45127, user_credentials={real_user=slave}} blocked reactor thread for 47949.4us
I20260926 07:35:38.147926 11402 leader_election.cc:302] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:38.149459 12381 raft_consensus.cc:2781] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:38.149777 12381 raft_consensus.cc:482] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:38.150060 12381 raft_consensus.cc:3037] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:38.156522 11746 tablet_bootstrap.cc:654] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:38.161904 12381 raft_consensus.cc:504] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:38.163512 12420 raft_consensus.cc:482] T c1e934f75d604d66aa12c9c8411d137a P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:38.163913 12420 raft_consensus.cc:504] T c1e934f75d604d66aa12c9c8411d137a P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.165910 12420 leader_election.cc:288] T c1e934f75d604d66aa12c9c8411d137a P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:38.169380 12381 leader_election.cc:288] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:38.181093 12354 raft_consensus.cc:482] T e5d942ed0a7b4f81bb9bfb915b5512f7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:38.181496 12354 raft_consensus.cc:504] T e5d942ed0a7b4f81bb9bfb915b5512f7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.183290 12354 leader_election.cc:288] T e5d942ed0a7b4f81bb9bfb915b5512f7 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:38.189990 11119 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 248ab84a1f284c2bbddf62f00b485e13 (table default.Table_8439aa0b11574c5b822c46f01f75d842 [id=47d4200172794cc78a6f386fee4b15b4]). Expected version 3 got 0
I20260926 07:35:38.198741 11746 tablet_bootstrap.cc:492] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:38.199182 11746 ts_tablet_manager.cc:1397] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.075s user 0.004s sys 0.004s
I20260926 07:35:38.205039 12432 tablet.cc:1722] T 0eae53ce278749208492815b918582f3 P 22f3902d01584936b20437606cb86dd1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 07:35:38.215631 12427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b6e2fc900b374ebab759831b571a208d)
I20260926 07:35:38.228346 12427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6ef5b900d0f4e73a358f415efa97249" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 37629 } } peers { permanent_uuid: "0773380643194748af6701300513e7a9" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 38823 } } peers { permanent_uuid: "b6e2fc900b374ebab759831b571a208d" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 40073 } }
I20260926 07:35:38.229703 11754 tablet_bootstrap.cc:492] T 0b1b14a340a7474fa8e76a464b0838e1 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:38.230084 11754 ts_tablet_manager.cc:1397] T 0b1b14a340a7474fa8e76a464b0838e1 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.194s user 0.017s sys 0.032s
I20260926 07:35:38.232085 11130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0773380643194748af6701300513e7a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 289 } } ignore_live_leader: false dest_uuid: "b6e2fc900b374ebab759831b571a208d" is_pre_election: true
I20260926 07:35:38.232841 11754 raft_consensus.cc:348] T 0b1b14a340a7474fa8e76a464b0838e1 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.234028 11754 raft_consensus.cc:374] T 0b1b14a340a7474fa8e76a464b0838e1 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:38.234272 11754 raft_consensus.cc:729] T 0b1b14a340a7474fa8e76a464b0838e1 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:38.234920 11754 consensus_queue.cc:260] T 0b1b14a340a7474fa8e76a464b0838e1 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
W20260926 07:35:38.245097 11402 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.171.3:45127, user_credentials={real_user=slave}} blocked reactor thread for 63525us
I20260926 07:35:38.249606 11754 ts_tablet_manager.cc:1428] T 0b1b14a340a7474fa8e76a464b0838e1 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.019s user 0.006s sys 0.001s
I20260926 07:35:38.250381 11754 tablet_bootstrap.cc:492] T 8983eb4db03c4073b94e205ac83ff098 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:38.254827 11754 tablet_bootstrap.cc:654] T 8983eb4db03c4073b94e205ac83ff098 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:38.245735 12259 ts_tablet_manager.cc:1918] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:38.255694 12259 log.cc:1196] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/c092aff8f2a041ccb09bc4b48e0f359c
I20260926 07:35:38.256654 12259 ts_tablet_manager.cc:1939] T c092aff8f2a041ccb09bc4b48e0f359c P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:38.258926 12259 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 171) took 3284 ms. Trace:
I20260926 07:35:38.231243 12427 leader_election.cc:288] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629), b6e2fc900b374ebab759831b571a208d (127.10.171.60:40073)
I20260926 07:35:38.265153 11871 ts_tablet_manager.cc:1918] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:35:38.265813 11871 log.cc:1196] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/6d524be8509645a89a75b4bac15f0889
I20260926 07:35:38.266395 11871 ts_tablet_manager.cc:1939] T 6d524be8509645a89a75b4bac15f0889 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:38.273815 11402 leader_election.cc:302] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:38.285971 12389 raft_consensus.cc:2781] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:38.286345 12389 raft_consensus.cc:482] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:38.286698 12389 raft_consensus.cc:3037] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:38.285270 12259 rpcz_store.cc:276] 0926 07:35:34.974366 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.974477 (+ 111us) service_pool.cc:227] Handling call
0926 07:35:37.698098 (+2723621us) ts_tablet_manager.cc:1145] Deleting tablet c092aff8f2a041ccb09bc4b48e0f359c
0926 07:35:37.698154 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:38.021057 (+322903us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:38.245492 (+224435us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:38.258865 (+ 13373us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2723139}
W20260926 07:35:38.293087 11545 leader_election.cc:341] T 6c42b518ef04429aa6b4e8f5deee05c8 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
W20260926 07:35:38.293939 11545 leader_election.cc:341] T 21919d22b1d24546911d36384b01d9cc P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:38.295291 11607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8f025d645eb478f95fa0aef17a86729" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:38.295785 11607 raft_consensus.cc:3037] T f8f025d645eb478f95fa0aef17a86729 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:38.297748 12068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0773380643194748af6701300513e7a9" candidate_term: 4 candidate_status { last_received { term: 3 index: 289 } } ignore_live_leader: false dest_uuid: "d6ef5b900d0f4e73a358f415efa97249" is_pre_election: true
I20260926 07:35:38.299532 11025 leader_election.cc:302] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0773380643194748af6701300513e7a9; no voters: b6e2fc900b374ebab759831b571a208d, d6ef5b900d0f4e73a358f415efa97249
I20260926 07:35:38.301162 12427 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0773380643194748af6701300513e7a9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260926 07:35:38.302213 11607 raft_consensus.cc:2454] T f8f025d645eb478f95fa0aef17a86729 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:38.275203 11746 raft_consensus.cc:348] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.303252 11746 raft_consensus.cc:374] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:38.303553 11746 raft_consensus.cc:729] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:38.304328 11746 consensus_queue.cc:260] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.306918 12389 raft_consensus.cc:504] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:38.309273 12389 leader_election.cc:288] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:38.311511 11604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e926ee35854c28a8c698384526fe38" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:38.312012 11604 raft_consensus.cc:3037] T b8e926ee35854c28a8c698384526fe38 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:38.321067 11607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17871fbd15064daba6e80a34a1b221d3" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:38.321651 11607 raft_consensus.cc:2454] T 17871fbd15064daba6e80a34a1b221d3 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
I20260926 07:35:38.329834 11604 raft_consensus.cc:2454] T b8e926ee35854c28a8c698384526fe38 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:38.268069 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 163) took 5459 ms. Trace:
I20260926 07:35:38.360533 11871 rpcz_store.cc:276] 0926 07:35:32.808857 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:32.810161 (+ 1304us) service_pool.cc:227] Handling call
0926 07:35:37.728440 (+4918279us) ts_tablet_manager.cc:1145] Deleting tablet 6d524be8509645a89a75b4bac15f0889
0926 07:35:37.728495 (+ 55us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:38.045711 (+317216us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:38.264959 (+219248us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:38.268020 (+ 3061us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4892708}
I20260926 07:35:38.361868 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 6d524be8509645a89a75b4bac15f0889 (table default.Table_f6f34cb43e5c462aa22636e6daaf274e [id=9fae0d37b0f8491ab8626e1f523e29ee]) successfully deleted
I20260926 07:35:38.364717 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet c092aff8f2a041ccb09bc4b48e0f359c (table default.Table_4c7ed90b32024fb888d7aef0e188ea80 [id=4e53b36526114bcabd8920c83fa2f6d4]) successfully deleted
I20260926 07:35:38.366852 11607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9f7faf55684e3e8512a033aa1443fe" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:38.367455 11607 raft_consensus.cc:3037] T ed9f7faf55684e3e8512a033aa1443fe P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:38.374706 11607 raft_consensus.cc:2454] T ed9f7faf55684e3e8512a033aa1443fe P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:38.331315 12259 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 172) took 3356 ms. Trace:
I20260926 07:35:38.375226 12259 rpcz_store.cc:276] 0926 07:35:34.975255 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.975340 (+ 85us) service_pool.cc:227] Handling call
0926 07:35:38.289464 (+3314124us) ts_tablet_manager.cc:1145] Deleting tablet f4aa8bef4bc64953ae2c573238838469
0926 07:35:38.330842 (+ 41378us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:38.331258 (+ 416us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3313648}
I20260926 07:35:38.376381 12259 tablet_replica.cc:330] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:38.377195 12259 raft_consensus.cc:2229] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:38.378028 12259 raft_consensus.cc:2258] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:38.333213 11746 ts_tablet_manager.cc:1428] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.134s user 0.004s sys 0.000s
I20260926 07:35:38.366012 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1e934f75d604d66aa12c9c8411d137a" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:38.366712 11465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5d942ed0a7b4f81bb9bfb915b5512f7" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:38.380282 11746 tablet_bootstrap.cc:492] T 6c42b518ef04429aa6b4e8f5deee05c8 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:38.386984 11444 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d524be8509645a89a75b4bac15f0889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:32 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:38.394156 11754 tablet_bootstrap.cc:492] T 8983eb4db03c4073b94e205ac83ff098 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:38.396862 12354 raft_consensus.cc:482] T 30df9b4afb414c90af5f4fb0c4582f01 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:38.418783 12354 raft_consensus.cc:504] T 30df9b4afb414c90af5f4fb0c4582f01 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.420746 12354 leader_election.cc:288] T 30df9b4afb414c90af5f4fb0c4582f01 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:38.422215 11754 ts_tablet_manager.cc:1397] T 8983eb4db03c4073b94e205ac83ff098 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.172s user 0.016s sys 0.023s
I20260926 07:35:38.424435 11754 raft_consensus.cc:348] T 8983eb4db03c4073b94e205ac83ff098 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.425132 11754 raft_consensus.cc:374] T 8983eb4db03c4073b94e205ac83ff098 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:38.425411 11754 raft_consensus.cc:729] T 8983eb4db03c4073b94e205ac83ff098 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:38.426262 11754 consensus_queue.cc:260] T 8983eb4db03c4073b94e205ac83ff098 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.406112 12440 tablet.cc:1722] T 0eae53ce278749208492815b918582f3 P 6070e9e3d1234a9d83e49431416afade: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 07:35:38.429003 11754 ts_tablet_manager.cc:1428] T 8983eb4db03c4073b94e205ac83ff098 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260926 07:35:38.429755 11754 tablet_bootstrap.cc:492] T 8392ebd0a4b84c2c94c3fe8745ea1ad7 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:38.405817 11871 tablet_replica.cc:330] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:38.430590 11871 raft_consensus.cc:2229] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:38.431541 11871 raft_consensus.cc:2258] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:38.434032 11871 ts_tablet_manager.cc:1905] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:38.437311 11118 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: default.Table_9158764a5a0d463ebc8413d234878ee7 [id=65c112d56f2a44cc9d840caadd218412] alter complete (version 2)
I20260926 07:35:38.396456 11402 leader_election.cc:302] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:38.438894 12429 raft_consensus.cc:2781] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:38.439309 12429 raft_consensus.cc:686] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:38.440052 12429 consensus_queue.cc:237] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:38.444543 11402 leader_election.cc:302] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:38.445235 12428 raft_consensus.cc:2781] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:38.445513 12428 raft_consensus.cc:482] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:38.445765 12428 raft_consensus.cc:3037] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:38.446105 11580 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d524be8509645a89a75b4bac15f0889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:32 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:38.447214 12354 raft_consensus.cc:482] T 5ee33141350b4c31aae79d2160e79b6f P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:38.447602 12354 raft_consensus.cc:504] T 5ee33141350b4c31aae79d2160e79b6f P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.448328 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet 21919d22b1d24546911d36384b01d9cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:29 UTC) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:38.449198 11402 leader_election.cc:302] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:38.449983 12434 raft_consensus.cc:2781] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:38.450378 12434 raft_consensus.cc:686] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:38.451081 12434 consensus_queue.cc:237] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:38.458899 12354 leader_election.cc:288] T 5ee33141350b4c31aae79d2160e79b6f P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:38.418516 12389 raft_consensus.cc:482] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:38.460013 12389 raft_consensus.cc:504] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.461721 12389 leader_election.cc:288] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:38.401645 12259 ts_tablet_manager.cc:1905] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:38.473701 11402 leader_election.cc:302] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:38.474427 12381 raft_consensus.cc:2781] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:38.474874 12381 raft_consensus.cc:686] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:38.475616 12381 consensus_queue.cc:237] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
W20260926 07:35:38.482544 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet f4aa8bef4bc64953ae2c573238838469 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f4aa8bef4bc64953ae2c573238838469 already in progress: creating tablet
W20260926 07:35:38.483485 11545 leader_election.cc:341] T c1e934f75d604d66aa12c9c8411d137a P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:38.485988 12389 raft_consensus.cc:482] T 3e28ca159d8349748812825ac1947c51 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6070e9e3d1234a9d83e49431416afade)
I20260926 07:35:38.486419 12389 raft_consensus.cc:504] T 3e28ca159d8349748812825ac1947c51 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.488903 12389 leader_election.cc:288] T 3e28ca159d8349748812825ac1947c51 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:38.490590 12428 raft_consensus.cc:504] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:38.492605 12428 leader_election.cc:288] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:38.508198 12432 tablet.cc:1722] T 248ab84a1f284c2bbddf62f00b485e13 P 22f3902d01584936b20437606cb86dd1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20260926 07:35:38.513173 11545 leader_election.cc:341] T e5d942ed0a7b4f81bb9bfb915b5512f7 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:38.514425 11607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21919d22b1d24546911d36384b01d9cc" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade" is_pre_election: true
I20260926 07:35:38.515208 11607 raft_consensus.cc:2454] T 21919d22b1d24546911d36384b01d9cc P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 0.
W20260926 07:35:38.515549 11104 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 5f2ef7ebe6e144529cffd699180d5be8 (table default.Table_034a885a3ed648e6a5df65b50925a7cd [id=b359877938204095b5ed3cfbe94997a3], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5f2ef7ebe6e144529cffd699180d5be8 consensus state has no leader
I20260926 07:35:38.521911 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17871fbd15064daba6e80a34a1b221d3" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:38.522434 11605 raft_consensus.cc:3037] T 17871fbd15064daba6e80a34a1b221d3 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:38.524725 11754 tablet_bootstrap.cc:654] T 8392ebd0a4b84c2c94c3fe8745ea1ad7 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:38.389813 11746 tablet_bootstrap.cc:654] T 6c42b518ef04429aa6b4e8f5deee05c8 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:38.527942 11871 ts_tablet_manager.cc:1918] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260926 07:35:38.515009 11463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30df9b4afb414c90af5f4fb0c4582f01" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:38.528275 11871 log.cc:1196] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/a0bdf6a4656e4217b0a7013fddc4cca8
I20260926 07:35:38.516294 11461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ee33141350b4c31aae79d2160e79b6f" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:38.528918 11871 ts_tablet_manager.cc:1939] T a0bdf6a4656e4217b0a7013fddc4cca8 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:38.530269 11605 raft_consensus.cc:2454] T 17871fbd15064daba6e80a34a1b221d3 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:38.534145 12251 mvcc.cc:205] Tried to move back new op lower bound from 7333511705692975104 to 7333511663350562816. Current Snapshot: MvccSnapshot[applied={T|T < 7333511705692975104 or (T in {7333511705692975104})}]
I20260926 07:35:38.537071 11402 leader_election.cc:302] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:38.538057 12389 raft_consensus.cc:2781] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260926 07:35:38.538127 11545 leader_election.cc:341] T 30df9b4afb414c90af5f4fb0c4582f01 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:38.538368 12389 raft_consensus.cc:482] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:38.538669 12389 raft_consensus.cc:3037] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Advancing to term 1
W20260926 07:35:38.538787 11545 leader_election.cc:341] T 5ee33141350b4c31aae79d2160e79b6f P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:38.548923 12450 tablet.cc:1722] T 248ab84a1f284c2bbddf62f00b485e13 P 6070e9e3d1234a9d83e49431416afade: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260926 07:35:38.552299 11754 tablet_bootstrap.cc:492] T 8392ebd0a4b84c2c94c3fe8745ea1ad7 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:38.553427 11754 ts_tablet_manager.cc:1397] T 8392ebd0a4b84c2c94c3fe8745ea1ad7 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.124s user 0.011s sys 0.003s
I20260926 07:35:38.554574 11118 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 5ee33141350b4c31aae79d2160e79b6f (table default.Table_86d0f0857c004fb8b506a9fde5aa3333 [id=295cf894a5064a209ee455ddbdfa528a]). Expected version 1 got 0
I20260926 07:35:38.555456 11118 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 248ab84a1f284c2bbddf62f00b485e13 (table default.Table_8439aa0b11574c5b822c46f01f75d842 [id=47d4200172794cc78a6f386fee4b15b4]). Expected version 3 got 0
I20260926 07:35:38.555526 11754 raft_consensus.cc:348] T 8392ebd0a4b84c2c94c3fe8745ea1ad7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.556183 11754 raft_consensus.cc:374] T 8392ebd0a4b84c2c94c3fe8745ea1ad7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:38.556414 11754 raft_consensus.cc:729] T 8392ebd0a4b84c2c94c3fe8745ea1ad7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:38.557053 11754 consensus_queue.cc:260] T 8392ebd0a4b84c2c94c3fe8745ea1ad7 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.575783 11754 ts_tablet_manager.cc:1428] T 8392ebd0a4b84c2c94c3fe8745ea1ad7 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20260926 07:35:38.583197 11754 tablet_bootstrap.cc:492] T f7459bf97b7f4b648d8ca6218cd72914 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:38.589974 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 164) took 5450 ms. Trace:
I20260926 07:35:38.590410 11871 rpcz_store.cc:276] 0926 07:35:33.139052 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:33.144705 (+ 5653us) service_pool.cc:227] Handling call
0926 07:35:38.362484 (+5217779us) ts_tablet_manager.cc:1145] Deleting tablet a0bdf6a4656e4217b0a7013fddc4cca8
0926 07:35:38.405655 (+ 43171us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:38.487540 (+ 81885us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:38.527755 (+ 40215us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:38.589902 (+ 62147us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5217200}
I20260926 07:35:38.591647 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 168) took 4985 ms. Trace:
I20260926 07:35:38.592867 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet a0bdf6a4656e4217b0a7013fddc4cca8 (table default.Table_1da003049e4b46eb994aaa3f3094d63c [id=3ce63f9fb5ad4b82ba2fbbf2e3678271]) successfully deleted
I20260926 07:35:38.591930 11871 rpcz_store.cc:276] 0926 07:35:33.606232 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:33.606351 (+ 119us) service_pool.cc:227] Handling call
0926 07:35:38.591350 (+4984999us) ts_tablet_manager.cc:1145] Deleting tablet 7019af1230224e999e078b21ea5a30a5
0926 07:35:38.591387 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:38.591603 (+ 216us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4984422}
W20260926 07:35:38.601138 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 7019af1230224e999e078b21ea5a30a5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7019af1230224e999e078b21ea5a30a5 already in progress: creating tablet
I20260926 07:35:38.602625 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 171) took 4873 ms. Trace:
I20260926 07:35:38.604087 11871 rpcz_store.cc:276] 0926 07:35:33.729164 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:33.729319 (+ 155us) service_pool.cc:227] Handling call
0926 07:35:38.602185 (+4872866us) ts_tablet_manager.cc:1145] Deleting tablet 4088ccb81e844cc29e1735bb341e4a92
0926 07:35:38.602237 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:38.602541 (+ 304us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4872325}
W20260926 07:35:38.609151 11402 leader_election.cc:341] T 3e28ca159d8349748812825ac1947c51 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Not found: Tablet not found: 3e28ca159d8349748812825ac1947c51
I20260926 07:35:38.609853 11402 leader_election.cc:302] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:38.611812 11746 tablet_bootstrap.cc:492] T 6c42b518ef04429aa6b4e8f5deee05c8 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:38.611969 11444 tablet_service.cc:1467] Processing CreateTablet for tablet 99f59e7d5e2c4489a71e3ceafd451e1b (DEFAULT_TABLE table=default.Table_6336482092624a0b849ec138f2e928bd [id=eaf43104ba664932b6c84a2d383bea02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:38.612885 11444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99f59e7d5e2c4489a71e3ceafd451e1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:38.614519 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet a0bdf6a4656e4217b0a7013fddc4cca8 with delete_type TABLET_DATA_DELETED (replaced by tablet 089bac5feeaa4508bc863e446b172033) from {username='slave'} at 127.0.0.1:35244
I20260926 07:35:38.614961 11580 tablet_service.cc:1514] Processing DeleteTablet for tablet 30df9b4afb414c90af5f4fb0c4582f01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:31 UTC) from {username='slave'} at 127.0.0.1:35244
W20260926 07:35:38.616403 11107 catalog_manager.cc:4821] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 4088ccb81e844cc29e1735bb341e4a92 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4088ccb81e844cc29e1735bb341e4a92 already in progress: creating tablet
I20260926 07:35:38.617449 11871 tablet_replica.cc:330] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:38.618489 11871 raft_consensus.cc:2229] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:38.620362 11871 raft_consensus.cc:2258] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:38.620486 12434 raft_consensus.cc:2781] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:38.598943 11754 tablet_bootstrap.cc:654] T f7459bf97b7f4b648d8ca6218cd72914 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:38.621061 12434 raft_consensus.cc:686] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:38.612195 11746 ts_tablet_manager.cc:1397] T 6c42b518ef04429aa6b4e8f5deee05c8 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.232s user 0.010s sys 0.000s
I20260926 07:35:38.621783 12434 consensus_queue.cc:237] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:38.623980 11871 ts_tablet_manager.cc:1905] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:38.624178 11746 raft_consensus.cc:348] T 6c42b518ef04429aa6b4e8f5deee05c8 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:38.624780 11746 raft_consensus.cc:374] T 6c42b518ef04429aa6b4e8f5deee05c8 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:38.625017 11746 raft_consensus.cc:729] T 6c42b518ef04429aa6b4e8f5deee05c8 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:38.625537 11746 consensus_queue.cc:260] T 6c42b518ef04429aa6b4e8f5deee05c8 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:38.630146 11580 tablet_service.cc:1467] Processing CreateTablet for tablet 99f59e7d5e2c4489a71e3ceafd451e1b (DEFAULT_TABLE table=default.Table_6336482092624a0b849ec138f2e928bd [id=eaf43104ba664932b6c84a2d383bea02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:38.631582 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99f59e7d5e2c4489a71e3ceafd451e1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:38.631254 12389 raft_consensus.cc:504] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.633292 11583 tablet_service.cc:1467] Processing CreateTablet for tablet d3e0272f2f7148cdbaa6575fe1befd34 (DEFAULT_TABLE table=default.Table_779bd60d3ede453ca6d35ead529d31fc [id=f10f8b1914e345d4a831510d90f0fb63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:38.634150 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e0272f2f7148cdbaa6575fe1befd34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:38.647533 12429 raft_consensus.cc:482] T 6d524be8509645a89a75b4bac15f0889 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:38.647969 12429 raft_consensus.cc:504] T 6d524be8509645a89a75b4bac15f0889 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:38.649766 12429 leader_election.cc:288] T 6d524be8509645a89a75b4bac15f0889 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127), e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829)
I20260926 07:35:38.651206 11746 ts_tablet_manager.cc:1428] T 6c42b518ef04429aa6b4e8f5deee05c8 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.029s user 0.005s sys 0.000s
I20260926 07:35:38.657702 11115 catalog_manager.cc:5495] T b8e926ee35854c28a8c698384526fe38 P 22f3902d01584936b20437606cb86dd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22f3902d01584936b20437606cb86dd1 (127.10.171.2). New cstate: current_term: 1 leader_uuid: "22f3902d01584936b20437606cb86dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260926 07:35:38.652004 11746 tablet_bootstrap.cc:492] T 25b8b701563344bbbc986db2b0f39b9b P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:38.659870 11115 catalog_manager.cc:5495] T f8f025d645eb478f95fa0aef17a86729 P 22f3902d01584936b20437606cb86dd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22f3902d01584936b20437606cb86dd1 (127.10.171.2). New cstate: current_term: 1 leader_uuid: "22f3902d01584936b20437606cb86dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260926 07:35:38.666500 11746 tablet_bootstrap.cc:654] T 25b8b701563344bbbc986db2b0f39b9b P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:38.669741 11754 tablet_bootstrap.cc:492] T f7459bf97b7f4b648d8ca6218cd72914 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:38.670131 11754 ts_tablet_manager.cc:1397] T f7459bf97b7f4b648d8ca6218cd72914 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.087s user 0.007s sys 0.001s
I20260926 07:35:38.672199 11754 raft_consensus.cc:348] T f7459bf97b7f4b648d8ca6218cd72914 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.672808 11754 raft_consensus.cc:374] T f7459bf97b7f4b648d8ca6218cd72914 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:38.673043 11754 raft_consensus.cc:729] T f7459bf97b7f4b648d8ca6218cd72914 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:38.689365 12389 leader_election.cc:288] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Requested vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127)
I20260926 07:35:38.698194 11754 consensus_queue.cc:260] T f7459bf97b7f4b648d8ca6218cd72914 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.701879 11754 ts_tablet_manager.cc:1428] T f7459bf97b7f4b648d8ca6218cd72914 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.032s user 0.006s sys 0.001s
I20260926 07:35:38.702517 11439 tablet_service.cc:1467] Processing CreateTablet for tablet d3e0272f2f7148cdbaa6575fe1befd34 (DEFAULT_TABLE table=default.Table_779bd60d3ede453ca6d35ead529d31fc [id=f10f8b1914e345d4a831510d90f0fb63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:38.703441 11439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e0272f2f7148cdbaa6575fe1befd34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:38.728048 11445 tablet_service.cc:1467] Processing CreateTablet for tablet e8d3a481998b4aeda3ef64c000c3aa26 (DEFAULT_TABLE table=default.Table_e9b22035d94d423280e51a4864e4aa84 [id=d19e4ba11fa14fa18c7ae93bf40f7c89]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 07:35:38.728859 11402 leader_election.cc:341] T 6d524be8509645a89a75b4bac15f0889 P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): Not found: Tablet not found: 6d524be8509645a89a75b4bac15f0889
I20260926 07:35:38.729035 11445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8d3a481998b4aeda3ef64c000c3aa26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:38.731961 12259 ts_tablet_manager.cc:1918] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:38.732273 12259 log.cc:1196] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/cef2e44be4084fc78e61cf822659b4ea
I20260926 07:35:38.732704 11435 tablet_service.cc:1467] Processing CreateTablet for tablet 6bbf6301e84448239d8a16d9f656a625 (DEFAULT_TABLE table=default.Table_74460676db784adf8c5d5ff8d76a021c [id=e1055de797e545dfad70276805f63008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:38.733503 11435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bbf6301e84448239d8a16d9f656a625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:38.733777 11115 catalog_manager.cc:5495] T ed9f7faf55684e3e8512a033aa1443fe P 22f3902d01584936b20437606cb86dd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22f3902d01584936b20437606cb86dd1 (127.10.171.2). New cstate: current_term: 1 leader_uuid: "22f3902d01584936b20437606cb86dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260926 07:35:38.738818 11444 tablet_service.cc:1467] Processing CreateTablet for tablet 1a2fa7c880b6497a9b79cb5d0309c41f (DEFAULT_TABLE table=default.Table_a178527b84ff446e95c6439e43798348 [id=1096af4470b04ab1aa715a3fe900578d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:38.739792 11444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a2fa7c880b6497a9b79cb5d0309c41f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:38.740290 11754 tablet_bootstrap.cc:492] T f4aa8bef4bc64953ae2c573238838469 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:38.745208 11754 tablet_bootstrap.cc:654] T f4aa8bef4bc64953ae2c573238838469 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:38.749328 11871 ts_tablet_manager.cc:1918] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:38.749687 11871 log.cc:1196] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/4239f5d48512403bb711ece401f0756b
I20260926 07:35:38.750356 11871 ts_tablet_manager.cc:1939] T 4239f5d48512403bb711ece401f0756b P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:38.752228 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 176) took 4665 ms. Trace:
I20260926 07:35:38.752537 11871 rpcz_store.cc:276] 0926 07:35:34.087226 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.087323 (+ 97us) service_pool.cc:227] Handling call
0926 07:35:38.617272 (+4529949us) ts_tablet_manager.cc:1145] Deleting tablet 4239f5d48512403bb711ece401f0756b
0926 07:35:38.617325 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:38.734903 (+117578us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:38.749026 (+ 14123us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:38.752174 (+ 3148us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4529437}
I20260926 07:35:38.766676 11746 tablet_bootstrap.cc:492] T 25b8b701563344bbbc986db2b0f39b9b P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:38.767088 11746 ts_tablet_manager.cc:1397] T 25b8b701563344bbbc986db2b0f39b9b P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.115s user 0.007s sys 0.000s
I20260926 07:35:38.732872 12259 ts_tablet_manager.cc:1939] T cef2e44be4084fc78e61cf822659b4ea P 22f3902d01584936b20437606cb86dd1: Deleting consensus metadata
I20260926 07:35:38.780474 11746 raft_consensus.cc:348] T 25b8b701563344bbbc986db2b0f39b9b P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:38.781117 11746 raft_consensus.cc:374] T 25b8b701563344bbbc986db2b0f39b9b P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:38.781522 11746 raft_consensus.cc:729] T 25b8b701563344bbbc986db2b0f39b9b P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:38.782159 11746 consensus_queue.cc:260] T 25b8b701563344bbbc986db2b0f39b9b P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } }
I20260926 07:35:38.784561 11746 ts_tablet_manager.cc:1428] T 25b8b701563344bbbc986db2b0f39b9b P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.017s user 0.007s sys 0.000s
I20260926 07:35:38.789112 12259 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 173) took 3640 ms. Trace:
I20260926 07:35:38.790992 12259 rpcz_store.cc:276] 0926 07:35:35.148493 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:35.356894 (+208401us) service_pool.cc:227] Handling call
0926 07:35:38.376217 (+3019323us) ts_tablet_manager.cc:1145] Deleting tablet cef2e44be4084fc78e61cf822659b4ea
0926 07:35:38.376272 (+ 55us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:38.499463 (+123191us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:38.731715 (+232252us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:38.789063 (+ 57348us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2976841}
I20260926 07:35:38.792435 12259 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 174) took 2678 ms. Trace:
I20260926 07:35:38.792781 12259 rpcz_store.cc:276] 0926 07:35:36.114089 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:36.114484 (+ 395us) service_pool.cc:227] Handling call
0926 07:35:38.792049 (+2677565us) ts_tablet_manager.cc:1145] Deleting tablet 0b900e55dcb34eef9f76de31b389e12d
0926 07:35:38.792097 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:38.792392 (+ 295us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2677117}
I20260926 07:35:38.795640 12259 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 175) took 2681 ms. Trace:
I20260926 07:35:38.797036 11105 catalog_manager.cc:4841] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): tablet cef2e44be4084fc78e61cf822659b4ea (table default.Table_ba602ce39d464e51bd0faa02b98f4363 [id=8a1a090674254074bc4db63f2c624eb8]) successfully deleted
W20260926 07:35:38.797782 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 0b900e55dcb34eef9f76de31b389e12d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0b900e55dcb34eef9f76de31b389e12d already in progress: creating tablet
I20260926 07:35:38.803753 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 4239f5d48512403bb711ece401f0756b (table default.Table_686ba040299440d0af09a9fe3501a040 [id=a639ca032a8d4fcdb928635eaa6db0e3]) successfully deleted
I20260926 07:35:38.805677 12259 rpcz_store.cc:276] 0926 07:35:36.114369 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:36.115013 (+ 644us) service_pool.cc:227] Handling call
0926 07:35:38.795318 (+2680305us) ts_tablet_manager.cc:1145] Deleting tablet 33ca5d33a5184ce3b4fe88a9a6755124
0926 07:35:38.795360 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:38.795599 (+ 239us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2679808}
I20260926 07:35:38.806326 11580 tablet_service.cc:1467] Processing CreateTablet for tablet 1a2fa7c880b6497a9b79cb5d0309c41f (DEFAULT_TABLE table=default.Table_a178527b84ff446e95c6439e43798348 [id=1096af4470b04ab1aa715a3fe900578d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:38.808074 11583 tablet_service.cc:1467] Processing CreateTablet for tablet e8d3a481998b4aeda3ef64c000c3aa26 (DEFAULT_TABLE table=default.Table_e9b22035d94d423280e51a4864e4aa84 [id=d19e4ba11fa14fa18c7ae93bf40f7c89]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 07:35:38.808389 11105 catalog_manager.cc:4821] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet 33ca5d33a5184ce3b4fe88a9a6755124 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33ca5d33a5184ce3b4fe88a9a6755124 already in progress: creating tablet
I20260926 07:35:38.810590 11589 tablet_service.cc:1467] Processing CreateTablet for tablet 6bbf6301e84448239d8a16d9f656a625 (DEFAULT_TABLE table=default.Table_74460676db784adf8c5d5ff8d76a021c [id=e1055de797e545dfad70276805f63008]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:35:38.812767 11746 tablet_bootstrap.cc:492] T 30df9b4afb414c90af5f4fb0c4582f01 P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:38.816582 11871 tablet_replica.cc:330] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:38.819133 11871 raft_consensus.cc:2229] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:38.824744 11118 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: default.Table_8439aa0b11574c5b822c46f01f75d842 [id=47d4200172794cc78a6f386fee4b15b4] alter complete (version 3)
I20260926 07:35:38.824939 11607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21919d22b1d24546911d36384b01d9cc" candidate_uuid: "22f3902d01584936b20437606cb86dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6070e9e3d1234a9d83e49431416afade"
I20260926 07:35:38.826022 11607 raft_consensus.cc:3037] T 21919d22b1d24546911d36384b01d9cc P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:38.828414 11746 tablet_bootstrap.cc:654] T 30df9b4afb414c90af5f4fb0c4582f01 P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:38.833022 11589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bbf6301e84448239d8a16d9f656a625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:38.833841 11871 raft_consensus.cc:2258] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:38.836546 11871 ts_tablet_manager.cc:1905] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:38.847596 12259 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 190) took 1281 ms. Trace:
I20260926 07:35:38.852191 12259 rpcz_store.cc:276] 0926 07:35:37.566029 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:37.566117 (+ 88us) service_pool.cc:227] Handling call
0926 07:35:38.847320 (+1281203us) ts_tablet_manager.cc:1145] Deleting tablet c81a59361a6e42b7962b3e027c4a087f
0926 07:35:38.847358 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:38.847518 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1280724}
W20260926 07:35:38.859956 11105 catalog_manager.cc:4808] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet c81a59361a6e42b7962b3e027c4a087f because the tablet was not found. No further retry: Not found: Tablet not found: c81a59361a6e42b7962b3e027c4a087f
I20260926 07:35:38.860309 12259 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40244 (request call id 191) took 1293 ms. Trace:
I20260926 07:35:38.860601 12259 rpcz_store.cc:276] 0926 07:35:37.566961 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:37.567056 (+ 95us) service_pool.cc:227] Handling call
0926 07:35:38.860043 (+1292987us) ts_tablet_manager.cc:1145] Deleting tablet a8b629a4aba045b697e33bbebecb05b7
0926 07:35:38.860081 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:38.860262 (+ 181us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1292551}
W20260926 07:35:38.861718 11105 catalog_manager.cc:4808] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): delete failed for tablet a8b629a4aba045b697e33bbebecb05b7 because the tablet was not found. No further retry: Not found: Tablet not found: a8b629a4aba045b697e33bbebecb05b7
I20260926 07:35:38.865084 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8d3a481998b4aeda3ef64c000c3aa26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:38.866746 12354 raft_consensus.cc:482] T 8392ebd0a4b84c2c94c3fe8745ea1ad7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:38.867244 12354 raft_consensus.cc:504] T 8392ebd0a4b84c2c94c3fe8745ea1ad7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.869488 12354 leader_election.cc:288] T 8392ebd0a4b84c2c94c3fe8745ea1ad7 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:38.870110 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a2fa7c880b6497a9b79cb5d0309c41f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:38.871490 11461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8392ebd0a4b84c2c94c3fe8745ea1ad7" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
I20260926 07:35:38.866238 11115 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d: default.Table_56493b3efdaa43e684afcd3d4b5dffdd [id=f66bffa952604b28bb51299de795ee2a] alter complete (version 1)
I20260926 07:35:38.876540 12259 tablet_replica.cc:330] T 9feaf1ed8ff34928b716fd0b4e41cdde P 22f3902d01584936b20437606cb86dd1: stopping tablet replica
I20260926 07:35:38.876641 11444 tablet_service.cc:1514] Processing DeleteTablet for tablet cef2e44be4084fc78e61cf822659b4ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:34 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:38.877303 12259 raft_consensus.cc:2229] T 9feaf1ed8ff34928b716fd0b4e41cdde P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:38.877898 12259 raft_consensus.cc:2258] T 9feaf1ed8ff34928b716fd0b4e41cdde P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 07:35:38.878463 11545 leader_election.cc:341] T 8392ebd0a4b84c2c94c3fe8745ea1ad7 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:38.889282 11444 tablet_service.cc:1514] Processing DeleteTablet for tablet 21919d22b1d24546911d36384b01d9cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 07:35:29 UTC) from {username='slave'} at 127.0.0.1:40244
I20260926 07:35:38.891736 11746 tablet_bootstrap.cc:492] T 30df9b4afb414c90af5f4fb0c4582f01 P 22f3902d01584936b20437606cb86dd1: No bootstrap required, opened a new log
I20260926 07:35:38.892153 11746 ts_tablet_manager.cc:1397] T 30df9b4afb414c90af5f4fb0c4582f01 P 22f3902d01584936b20437606cb86dd1: Time spent bootstrapping tablet: real 0.080s user 0.017s sys 0.013s
I20260926 07:35:38.894205 11746 raft_consensus.cc:348] T 30df9b4afb414c90af5f4fb0c4582f01 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.894793 11746 raft_consensus.cc:374] T 30df9b4afb414c90af5f4fb0c4582f01 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:38.895015 11746 raft_consensus.cc:729] T 30df9b4afb414c90af5f4fb0c4582f01 P 22f3902d01584936b20437606cb86dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Initialized, Role: FOLLOWER
I20260926 07:35:38.903080 11118 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet f7459bf97b7f4b648d8ca6218cd72914 (table default.Table_b61b46186a5a49bdba86babfd064d3ab [id=e3d379a69ef94c6a990d3dfbea076591]). Expected version 1 got 0
I20260926 07:35:38.904372 11746 consensus_queue.cc:260] T 30df9b4afb414c90af5f4fb0c4582f01 P 22f3902d01584936b20437606cb86dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.909835 11607 raft_consensus.cc:2454] T 21919d22b1d24546911d36384b01d9cc P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22f3902d01584936b20437606cb86dd1 in term 1.
I20260926 07:35:38.910880 11402 leader_election.cc:302] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22f3902d01584936b20437606cb86dd1, 6070e9e3d1234a9d83e49431416afade; no voters:
I20260926 07:35:38.911572 12434 raft_consensus.cc:2781] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:38.911952 12434 raft_consensus.cc:686] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 22f3902d01584936b20437606cb86dd1, State: Running, Role: LEADER
I20260926 07:35:38.912173 12354 raft_consensus.cc:482] T 8983eb4db03c4073b94e205ac83ff098 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:38.912613 12354 raft_consensus.cc:504] T 8983eb4db03c4073b94e205ac83ff098 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.912690 12434 consensus_queue.cc:237] T 21919d22b1d24546911d36384b01d9cc P 22f3902d01584936b20437606cb86dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.913836 11754 tablet_bootstrap.cc:492] T f4aa8bef4bc64953ae2c573238838469 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:38.914292 11754 ts_tablet_manager.cc:1397] T f4aa8bef4bc64953ae2c573238838469 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.212s user 0.005s sys 0.013s
I20260926 07:35:38.914544 12354 leader_election.cc:288] T 8983eb4db03c4073b94e205ac83ff098 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
I20260926 07:35:38.916368 11461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8983eb4db03c4073b94e205ac83ff098" candidate_uuid: "6070e9e3d1234a9d83e49431416afade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22f3902d01584936b20437606cb86dd1" is_pre_election: true
W20260926 07:35:38.917654 11545 leader_election.cc:341] T 8983eb4db03c4073b94e205ac83ff098 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:35:38.918149 11746 ts_tablet_manager.cc:1428] T 30df9b4afb414c90af5f4fb0c4582f01 P 22f3902d01584936b20437606cb86dd1: Time spent starting tablet: real 0.026s user 0.004s sys 0.001s
I20260926 07:35:38.918934 11746 tablet_bootstrap.cc:492] T c1e934f75d604d66aa12c9c8411d137a P 22f3902d01584936b20437606cb86dd1: Bootstrap starting.
I20260926 07:35:38.920491 12259 ts_tablet_manager.cc:1905] T 9feaf1ed8ff34928b716fd0b4e41cdde P 22f3902d01584936b20437606cb86dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:38.925608 11754 raft_consensus.cc:348] T f4aa8bef4bc64953ae2c573238838469 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.926231 11754 raft_consensus.cc:374] T f4aa8bef4bc64953ae2c573238838469 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:38.926474 11754 raft_consensus.cc:729] T f4aa8bef4bc64953ae2c573238838469 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:38.926985 11754 consensus_queue.cc:260] T f4aa8bef4bc64953ae2c573238838469 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:38.931658 11871 ts_tablet_manager.cc:1918] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:38.932031 11871 log.cc:1196] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/04be706330924282961054c5835cec73
I20260926 07:35:38.933451 11871 ts_tablet_manager.cc:1939] T 04be706330924282961054c5835cec73 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:38.935856 11746 tablet_bootstrap.cc:654] T c1e934f75d604d66aa12c9c8411d137a P 22f3902d01584936b20437606cb86dd1: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:38.966513 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 177) took 4865 ms. Trace:
I20260926 07:35:38.966813 11871 rpcz_store.cc:276] 0926 07:35:34.101590 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.101680 (+ 90us) service_pool.cc:227] Handling call
0926 07:35:38.753456 (+4651776us) ts_tablet_manager.cc:1145] Deleting tablet 04be706330924282961054c5835cec73
0926 07:35:38.815997 (+ 62541us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:38.919374 (+103377us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:38.931476 (+ 12102us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:38.966446 (+ 34970us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":31128,"tablet-delete.queue_time_us":4650132}
I20260926 07:35:38.967517 12259 ts_tablet_manager.cc:1918] T 9feaf1ed8ff34928b716fd0b4e41cdde P 22f3902d01584936b20437606cb86dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:38.967867 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 04be706330924282961054c5835cec73 (table default.Table_4eb22a54d0df4f808fe85886a5809ab7 [id=cab439115f3c46d089824983fe26d0eb]) successfully deleted
W20260926 07:35:38.968089 11546 connection.cc:535] server connection from 127.10.171.1:38217 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:35:38.968743 11546 connection.cc:535] client connection to 127.10.171.1:41829 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:35:38.969157 11546 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 07:35:38.969990 11754 ts_tablet_manager.cc:1428] T f4aa8bef4bc64953ae2c573238838469 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.055s user 0.004s sys 0.003s
I20260926 07:35:38.972519 11871 tablet_replica.cc:330] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
W20260926 07:35:38.974756 11106 connection.cc:535] client connection to 127.10.171.1:41829 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:35:38.975082 11106 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 07:35:38.978760 11118 catalog_manager.cc:5495] T 17871fbd15064daba6e80a34a1b221d3 P 22f3902d01584936b20437606cb86dd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22f3902d01584936b20437606cb86dd1 (127.10.171.2). New cstate: current_term: 1 leader_uuid: "22f3902d01584936b20437606cb86dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } health_report { overall_health: UNKNOWN } } }
W20260926 07:35:38.984717 11105 connection.cc:535] server connection from 127.10.171.1:34359 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 07:35:38.989053 11118 catalog_manager.cc:5587] TS 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379) does not have the latest schema for tablet 6c42b518ef04429aa6b4e8f5deee05c8 (table default.Table_3de17a801f1c4fa99878e07e9df543f8 [id=0ebbd3b8ecd242c2a21e0630227a01c7]). Expected version 2 got 0
I20260926 07:35:38.992547 10924 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu with pid 11369
I20260926 07:35:39.002581 11754 tablet_bootstrap.cc:492] T 16ab928a487c4f499aba3b366d01d89b P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:39.015693 11871 raft_consensus.cc:2229] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:35:39.016628 11871 raft_consensus.cc:2258] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:39.018433 11871 ts_tablet_manager.cc:1905] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 07:35:39.023368 11546 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.1:41829, user_credentials={real_user=slave}} blocked reactor thread for 54299.2us
W20260926 07:35:39.024837 11106 catalog_manager.cc:4584] TS e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): CreateTablet RPC failed for tablet 361ab96150254adaa58eb02e1dba0300: Network error: Client connection negotiation failed: client connection to 127.10.171.1:41829: connect: Connection refused (error 111)
I20260926 07:35:39.030045 11754 tablet_bootstrap.cc:654] T 16ab928a487c4f499aba3b366d01d89b P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:39.036384 11546 consensus_peers.cc:487] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Status: Network error: Client connection negotiation failed: client connection to 127.10.171.1:41829: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260926 07:35:39.036978 11546 leader_election.cc:334] T 6c42b518ef04429aa6b4e8f5deee05c8 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Network error: Client connection negotiation failed: client connection to 127.10.171.1:41829: connect: Connection refused (error 111)
I20260926 07:35:39.037448 11546 leader_election.cc:302] T 6c42b518ef04429aa6b4e8f5deee05c8 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:39.038323 12354 raft_consensus.cc:2726] T 6c42b518ef04429aa6b4e8f5deee05c8 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:35:39.038554 11546 leader_election.cc:334] T 21919d22b1d24546911d36384b01d9cc P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Network error: Client connection negotiation failed: client connection to 127.10.171.1:41829: connect: Connection refused (error 111)
I20260926 07:35:39.038986 11546 leader_election.cc:302] T 21919d22b1d24546911d36384b01d9cc P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:39.039844 12354 raft_consensus.cc:2726] T 21919d22b1d24546911d36384b01d9cc P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:35:39.040067 11546 leader_election.cc:334] T c1e934f75d604d66aa12c9c8411d137a P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Network error: Client connection negotiation failed: client connection to 127.10.171.1:41829: connect: Connection refused (error 111)
I20260926 07:35:39.040510 11546 leader_election.cc:302] T c1e934f75d604d66aa12c9c8411d137a P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:39.041213 12354 raft_consensus.cc:2726] T c1e934f75d604d66aa12c9c8411d137a P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:35:39.041534 11546 leader_election.cc:334] T e5d942ed0a7b4f81bb9bfb915b5512f7 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Network error: Client connection negotiation failed: client connection to 127.10.171.1:41829: connect: Connection refused (error 111)
I20260926 07:35:39.041934 11546 leader_election.cc:302] T e5d942ed0a7b4f81bb9bfb915b5512f7 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:39.042714 12354 raft_consensus.cc:2726] T e5d942ed0a7b4f81bb9bfb915b5512f7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:35:39.043035 11546 consensus_peers.cc:487] T 4826318c83524f41840e770f33bd3696 P 6070e9e3d1234a9d83e49431416afade -> Peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Couldn't send request to peer e9bfebb34fa14cd2ad4843fffbe5c54e. Status: Network error: Client connection negotiation failed: client connection to 127.10.171.1:41829: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260926 07:35:39.043665 11546 leader_election.cc:334] T 30df9b4afb414c90af5f4fb0c4582f01 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Network error: Client connection negotiation failed: client connection to 127.10.171.1:41829: connect: Connection refused (error 111)
I20260926 07:35:39.044044 11546 leader_election.cc:302] T 30df9b4afb414c90af5f4fb0c4582f01 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:39.044770 12354 raft_consensus.cc:2726] T 30df9b4afb414c90af5f4fb0c4582f01 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:35:39.044939 11546 leader_election.cc:334] T 5ee33141350b4c31aae79d2160e79b6f P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Network error: Client connection negotiation failed: client connection to 127.10.171.1:41829: connect: Connection refused (error 111)
I20260926 07:35:39.045291 11546 leader_election.cc:302] T 5ee33141350b4c31aae79d2160e79b6f P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:39.045928 12354 raft_consensus.cc:2726] T 5ee33141350b4c31aae79d2160e79b6f P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:35:39.046046 11546 leader_election.cc:334] T 8392ebd0a4b84c2c94c3fe8745ea1ad7 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Network error: Client connection negotiation failed: client connection to 127.10.171.1:41829: connect: Connection refused (error 111)
I20260926 07:35:39.046418 11546 leader_election.cc:302] T 8392ebd0a4b84c2c94c3fe8745ea1ad7 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:39.047026 12354 raft_consensus.cc:2726] T 8392ebd0a4b84c2c94c3fe8745ea1ad7 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:35:39.047183 11546 leader_election.cc:334] T 8983eb4db03c4073b94e205ac83ff098 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Network error: Client connection negotiation failed: client connection to 127.10.171.1:41829: connect: Connection refused (error 111)
I20260926 07:35:39.047520 11546 leader_election.cc:302] T 8983eb4db03c4073b94e205ac83ff098 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:39.048230 12354 raft_consensus.cc:2726] T 8983eb4db03c4073b94e205ac83ff098 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:35:39.057843 11754 tablet_bootstrap.cc:492] T 16ab928a487c4f499aba3b366d01d89b P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:39.058219 11754 ts_tablet_manager.cc:1397] T 16ab928a487c4f499aba3b366d01d89b P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.056s user 0.008s sys 0.000s
I20260926 07:35:39.060415 11754 raft_consensus.cc:348] T 16ab928a487c4f499aba3b366d01d89b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:39.061039 11754 raft_consensus.cc:374] T 16ab928a487c4f499aba3b366d01d89b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:39.061271 11754 raft_consensus.cc:729] T 16ab928a487c4f499aba3b366d01d89b P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:39.061831 11754 consensus_queue.cc:260] T 16ab928a487c4f499aba3b366d01d89b P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:39.090822 11754 ts_tablet_manager.cc:1428] T 16ab928a487c4f499aba3b366d01d89b P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.032s user 0.006s sys 0.001s
I20260926 07:35:39.093649 11871 ts_tablet_manager.cc:1918] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:39.094193 11871 log.cc:1196] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/c36c759963cb4915afdca7d12109cc6e
I20260926 07:35:39.094827 11871 ts_tablet_manager.cc:1939] T c36c759963cb4915afdca7d12109cc6e P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:39.098361 12354 raft_consensus.cc:482] T 0b1b14a340a7474fa8e76a464b0838e1 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:39.098501 11118 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 16ab928a487c4f499aba3b366d01d89b (table default.Table_3a3f3643ccfd4c82a0a33fcaed606b63 [id=f9582c45829d4f10af98e245dd41a907]). Expected version 1 got 0
I20260926 07:35:39.098750 12354 raft_consensus.cc:504] T 0b1b14a340a7474fa8e76a464b0838e1 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:39.100488 12354 leader_election.cc:288] T 0b1b14a340a7474fa8e76a464b0838e1 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829), 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379)
W20260926 07:35:39.103641 11546 leader_election.cc:334] T 0b1b14a340a7474fa8e76a464b0838e1 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer e9bfebb34fa14cd2ad4843fffbe5c54e (127.10.171.1:41829): Network error: Client connection negotiation failed: client connection to 127.10.171.1:41829: connect: Connection refused (error 111)
I20260926 07:35:39.110450 11754 tablet_bootstrap.cc:492] T 89ddc16f2635473da43fe21676a947fa P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:39.124380 11118 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 16ab928a487c4f499aba3b366d01d89b (table default.Table_3a3f3643ccfd4c82a0a33fcaed606b63 [id=f9582c45829d4f10af98e245dd41a907]). Expected version 1 got 0
I20260926 07:35:39.128279 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 179) took 5015 ms. Trace:
I20260926 07:35:39.129240 11754 tablet_bootstrap.cc:654] T 89ddc16f2635473da43fe21676a947fa P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:39.128558 11871 rpcz_store.cc:276] 0926 07:35:34.112737 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.114003 (+ 1266us) service_pool.cc:227] Handling call
0926 07:35:38.972327 (+4858324us) ts_tablet_manager.cc:1145] Deleting tablet c36c759963cb4915afdca7d12109cc6e
0926 07:35:38.972396 (+ 69us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:39.048968 (+ 76572us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:39.093456 (+ 44488us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:39.128220 (+ 34764us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4857755}
I20260926 07:35:39.130861 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet c36c759963cb4915afdca7d12109cc6e (table default.Table_b9f92e070aab45cdb8352c1d8a3d603c [id=7245a652e96a445fa9ae35517ac03adf]) successfully deleted
I20260926 07:35:39.132434 11871 tablet_replica.cc:330] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:39.133177 11871 raft_consensus.cc:2229] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:39.133576 11871 raft_consensus.cc:2258] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:35:39.136144 11871 ts_tablet_manager.cc:1905] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:35:39.155048 11754 tablet_bootstrap.cc:492] T 89ddc16f2635473da43fe21676a947fa P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:39.155442 11754 ts_tablet_manager.cc:1397] T 89ddc16f2635473da43fe21676a947fa P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.045s user 0.006s sys 0.003s
I20260926 07:35:39.157647 11754 raft_consensus.cc:348] T 89ddc16f2635473da43fe21676a947fa P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:39.158252 11754 raft_consensus.cc:374] T 89ddc16f2635473da43fe21676a947fa P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:39.158486 11754 raft_consensus.cc:729] T 89ddc16f2635473da43fe21676a947fa P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:39.159042 11754 consensus_queue.cc:260] T 89ddc16f2635473da43fe21676a947fa P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:39.161079 11754 ts_tablet_manager.cc:1428] T 89ddc16f2635473da43fe21676a947fa P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.005s user 0.003s sys 0.004s
I20260926 07:35:39.166059 11871 ts_tablet_manager.cc:1918] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:35:39.168730 11754 tablet_bootstrap.cc:492] T 393da59b3671473ca9714d3406dd1388 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:39.173615 11754 tablet_bootstrap.cc:654] T 393da59b3671473ca9714d3406dd1388 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:39.188129 11118 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 89ddc16f2635473da43fe21676a947fa (table default.Table_d0380e9d3f26439bbc91cf476aeae69f [id=19875953f2b6484f934503404af57593]). Expected version 2 got 0
W20260926 07:35:39.190526 11118 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet 89ddc16f2635473da43fe21676a947fa (table default.Table_d0380e9d3f26439bbc91cf476aeae69f [id=19875953f2b6484f934503404af57593], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 89ddc16f2635473da43fe21676a947fa consensus state has no leader
I20260926 07:35:39.210438 11754 tablet_bootstrap.cc:492] T 393da59b3671473ca9714d3406dd1388 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:39.210960 11754 ts_tablet_manager.cc:1397] T 393da59b3671473ca9714d3406dd1388 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.043s user 0.008s sys 0.001s
I20260926 07:35:39.166385 11871 log.cc:1196] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/5cd482fb159d418291cb02bc9138b076
I20260926 07:35:39.211728 11871 ts_tablet_manager.cc:1939] T 5cd482fb159d418291cb02bc9138b076 P 6070e9e3d1234a9d83e49431416afade: Deleting consensus metadata
I20260926 07:35:39.213397 11754 raft_consensus.cc:348] T 393da59b3671473ca9714d3406dd1388 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:39.214151 11754 raft_consensus.cc:374] T 393da59b3671473ca9714d3406dd1388 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:39.214416 11754 raft_consensus.cc:729] T 393da59b3671473ca9714d3406dd1388 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
I20260926 07:35:39.214473 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 187) took 4636 ms. Trace:
I20260926 07:35:39.214813 11871 rpcz_store.cc:276] 0926 07:35:34.577977 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.578109 (+ 132us) service_pool.cc:227] Handling call
0926 07:35:39.132259 (+4554150us) ts_tablet_manager.cc:1145] Deleting tablet 5cd482fb159d418291cb02bc9138b076
0926 07:35:39.132316 (+ 57us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:39.142872 (+ 10556us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:39.165875 (+ 23003us) tablet_metadata.cc:641] Metadata flushed
0926 07:35:39.214420 (+ 48545us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4553533}
I20260926 07:35:39.215071 11754 consensus_queue.cc:260] T 393da59b3671473ca9714d3406dd1388 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:39.216183 11107 catalog_manager.cc:4841] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): tablet 5cd482fb159d418291cb02bc9138b076 (table default.Table_54b98a70017749499ab6c312bd7e6255 [id=3895daba9d194983a7018f02aa7b02a5]) successfully deleted
I20260926 07:35:39.216656 11118 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 89ddc16f2635473da43fe21676a947fa (table default.Table_d0380e9d3f26439bbc91cf476aeae69f [id=19875953f2b6484f934503404af57593]). Expected version 2 got 0
I20260926 07:35:39.226387 11754 ts_tablet_manager.cc:1428] T 393da59b3671473ca9714d3406dd1388 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260926 07:35:39.227272 11754 tablet_bootstrap.cc:492] T 854ac84f21304d57818dd052464127b8 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:39.254218 11754 tablet_bootstrap.cc:654] T 854ac84f21304d57818dd052464127b8 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:39.260160 11871 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35244 (request call id 186) took 4682 ms. Trace:
I20260926 07:35:39.260545 11871 rpcz_store.cc:276] 0926 07:35:34.577594 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 07:35:34.578899 (+ 1305us) service_pool.cc:227] Handling call
0926 07:35:39.259830 (+4680931us) ts_tablet_manager.cc:1145] Deleting tablet 9feaf1ed8ff34928b716fd0b4e41cdde
0926 07:35:39.259890 (+ 60us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0926 07:35:39.260090 (+ 200us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4680281}
W20260926 07:35:39.261739 11107 catalog_manager.cc:4808] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127): delete failed for tablet 9feaf1ed8ff34928b716fd0b4e41cdde because the tablet was not found. No further retry: Not found: Tablet not found: 9feaf1ed8ff34928b716fd0b4e41cdde
I20260926 07:35:39.262761 11871 tablet_replica.cc:330] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade: stopping tablet replica
I20260926 07:35:39.263634 11871 raft_consensus.cc:2229] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:35:39.266045 11754 tablet_bootstrap.cc:492] T 854ac84f21304d57818dd052464127b8 P 6070e9e3d1234a9d83e49431416afade: No bootstrap required, opened a new log
I20260926 07:35:39.266409 11754 ts_tablet_manager.cc:1397] T 854ac84f21304d57818dd052464127b8 P 6070e9e3d1234a9d83e49431416afade: Time spent bootstrapping tablet: real 0.039s user 0.014s sys 0.000s
I20260926 07:35:39.268643 11118 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 393da59b3671473ca9714d3406dd1388 (table default.Table_54b98a70017749499ab6c312bd7e6255 [id=3895daba9d194983a7018f02aa7b02a5]). Expected version 1 got 0
I20260926 07:35:39.268635 11754 raft_consensus.cc:348] T 854ac84f21304d57818dd052464127b8 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:39.269366 11754 raft_consensus.cc:374] T 854ac84f21304d57818dd052464127b8 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:39.269642 11754 raft_consensus.cc:729] T 854ac84f21304d57818dd052464127b8 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6070e9e3d1234a9d83e49431416afade, State: Initialized, Role: FOLLOWER
W20260926 07:35:39.270100 11118 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet 393da59b3671473ca9714d3406dd1388 (table default.Table_54b98a70017749499ab6c312bd7e6255 [id=3895daba9d194983a7018f02aa7b02a5], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 393da59b3671473ca9714d3406dd1388 consensus state has no leader
I20260926 07:35:39.270238 11754 consensus_queue.cc:260] T 854ac84f21304d57818dd052464127b8 P 6070e9e3d1234a9d83e49431416afade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9bfebb34fa14cd2ad4843fffbe5c54e" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 41829 } } peers { permanent_uuid: "22f3902d01584936b20437606cb86dd1" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 42379 } } peers { permanent_uuid: "6070e9e3d1234a9d83e49431416afade" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 45127 } }
I20260926 07:35:39.276363 11754 ts_tablet_manager.cc:1428] T 854ac84f21304d57818dd052464127b8 P 6070e9e3d1234a9d83e49431416afade: Time spent starting tablet: real 0.010s user 0.006s sys 0.001s
I20260926 07:35:39.277148 11754 tablet_bootstrap.cc:492] T 6d54a5c32da5444ca0a2939461f3a0f6 P 6070e9e3d1234a9d83e49431416afade: Bootstrap starting.
I20260926 07:35:39.281893 11754 tablet_bootstrap.cc:654] T 6d54a5c32da5444ca0a2939461f3a0f6 P 6070e9e3d1234a9d83e49431416afade: Neither blocks nor log segments found. Creating new log.
W20260926 07:35:39.301476 11988 connection.cc:535] server connection from 127.10.171.2:54119 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:35:39.302922 11545 connection.cc:535] client connection to 127.10.171.2:42379 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 07:35:39.306465 11871 raft_consensus.cc:2258] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 07:35:39.311836 11105 connection.cc:535] server connection from 127.10.171.2:33317 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:35:39.312780 11105 connection.cc:535] client connection to 127.10.171.2:42379 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 07:35:39.314915 11118 catalog_manager.cc:5587] TS 6070e9e3d1234a9d83e49431416afade (127.10.171.3:45127) does not have the latest schema for tablet 854ac84f21304d57818dd052464127b8 (table default.Table_52e8769a2e55412eb074de5c4f541ba6 [id=83482df6f10341778ca79919c84c5fc8]). Expected version 1 got 0
I20260926 07:35:39.315976 11871 ts_tablet_manager.cc:1905] T c092aff8f2a041ccb09bc4b48e0f359c P 6070e9e3d1234a9d83e49431416afade: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 07:35:39.316289 12466 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:35:39.304564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.171.2:42379 (local address 127.10.171.3:43839)
0926 07:35:39.306033 (+ 1469us) negotiation.cc:107] Waiting for socket to connect
0926 07:35:39.306096 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0926 07:35:39.306486 (+ 390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 07:35:39.315259 (+ 8773us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.171.2:42379: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1041}
W20260926 07:35:39.317258 11545 leader_election.cc:334] T 0b1b14a340a7474fa8e76a464b0838e1 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Network error: Client connection negotiation failed: client connection to 127.10.171.2:42379: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 07:35:39.317600 11545 leader_election.cc:302] T 0b1b14a340a7474fa8e76a464b0838e1 P 6070e9e3d1234a9d83e49431416afade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6070e9e3d1234a9d83e49431416afade; no voters: 22f3902d01584936b20437606cb86dd1, e9bfebb34fa14cd2ad4843fffbe5c54e
I20260926 07:35:39.320272 12354 raft_consensus.cc:2726] T 0b1b14a340a7474fa8e76a464b0838e1 P 6070e9e3d1234a9d83e49431416afade [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:35:39.322283 11545 consensus_peers.cc:487] T 7e52ec7d802a4e90bccef777f77a2a1b P 6070e9e3d1234a9d83e49431416afade -> Peer 22f3902d01584936b20437606cb86dd1 (127.10.171.2:42379): Couldn't send request to peer 22f3902d01584936b20437606cb86dd1. Status: Network error: Client connection negotiation failed: client connection to 127.10.171.2:42379: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:39.324085 10924 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu with pid 11521
I20260926 07:35:39.412849 10924 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu with pid 11716
W20260926 07:35:39.414402 11106 connection.cc:535] server connection from 127.10.171.3:53335 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 07:35:39.469530 10924 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu with pid 11003
W20260926 07:35:39.485628 11104 consensus_peers.cc:487] T 00000000000000000000000000000000 P b6e2fc900b374ebab759831b571a208d -> Peer d6ef5b900d0f4e73a358f415efa97249 (127.10.171.62:37629): Couldn't send request to peer d6ef5b900d0f4e73a358f415efa97249. Status: Network error: Client connection negotiation failed: client connection to 127.10.171.62:37629: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 07:35:39.520485 10924 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu with pid 11078
2026-09-26T07:35:39Z chronyd exiting
I20260926 07:35:39.585492 10924 master-stress-test.cc:586] Tables created: 117
I20260926 07:35:39.585754 10924 master-stress-test.cc:587] Tables altered: 58
I20260926 07:35:39.585908 10924 master-stress-test.cc:588] Tables deleted: 46
I20260926 07:35:39.586063 10924 master-stress-test.cc:589] Tablets replaced: 20
I20260926 07:35:39.586239 10924 master-stress-test.cc:590] Masters restarted: 1
[ OK ] CatalogManagerConfigurations/MasterStressTest.Test/0 (33553 ms)
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/3
I20260926 07:35:39.646579 10924 test_util.cc:268] Using random seed: -517298596
2026-09-26T07:35:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T07:35:39Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskNvrOpy/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskNvrOpy/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/security.properties
2026-09-26 07:35:45: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskNvrOpy/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskNvrOpy/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260926 07:35:53.352777 10924 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
/tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.171.62:36693
--webserver_interface=127.10.171.62
--webserver_port=0
--builtin_ntp_servers=127.10.171.20:42033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615
--hive_metastore_uris=thrift://127.0.0.1:35621
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 07:35:53.659660 12713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:35:53.660534 12713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:35:53.661084 12713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:35:53.691907 12713 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260926 07:35:53.692227 12713 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260926 07:35:53.692508 12713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:35:53.692767 12713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:35:53.693033 12713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:35:53.693289 12713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:35:53.728026 12713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.171.20:42033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:35621
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.171.62:36693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.171.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:55:20 UTC on ccycloud.quasar-qwhbmo.root.comops.site
build id 4673737
TSAN enabled
I20260926 07:35:53.729360 12713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:35:53.730911 12713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:35:53.747994 12719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:35:53.749577 12713 server_base.cc:1034] running on GCE node
W20260926 07:35:53.748080 12722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:35:53.748049 12720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:35:54.928316 12713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:35:54.931381 12713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:35:54.932817 12713 hybrid_clock.cc:648] HybridClock initialized: now 1790408154932740 us; error 86 us; skew 500 ppm
I20260926 07:35:54.933632 12713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:35:54.947271 12713 webserver.cc:460] Webserver started at http://127.10.171.62:32999/ using document root <none> and password file <none>
I20260926 07:35:54.948498 12713 fs_manager.cc:359] Metadata directory not provided
I20260926 07:35:54.948724 12713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:35:54.949150 12713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:35:54.955498 12713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-0/data/instance:
uuid: "51342b641d47457d84f48fcbb6c48375"
format_stamp: "Formatted at 2026-09-26 07:35:54 on dist-test-slave-3mmx"
I20260926 07:35:54.956619 12713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-0/wal/instance:
uuid: "51342b641d47457d84f48fcbb6c48375"
format_stamp: "Formatted at 2026-09-26 07:35:54 on dist-test-slave-3mmx"
I20260926 07:35:54.963966 12713 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260926 07:35:54.969251 12729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:35:54.970386 12713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260926 07:35:54.970711 12713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-0/data,/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-0/wal
uuid: "51342b641d47457d84f48fcbb6c48375"
format_stamp: "Formatted at 2026-09-26 07:35:54 on dist-test-slave-3mmx"
I20260926 07:35:54.971056 12713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:35:55.007194 12713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:35:55.008728 12713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:35:55.009202 12713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:35:55.078058 12713 rpc_server.cc:307] RPC server started. Bound to: 127.10.171.62:36693
I20260926 07:35:55.078164 12780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.171.62:36693 every 8 connection(s)
I20260926 07:35:55.080725 12713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-0/data/info.pb
I20260926 07:35:55.085609 12781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:55.086269 10924 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu as pid 12713
I20260926 07:35:55.086776 10924 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-0/wal/instance
I20260926 07:35:55.090694 10924 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
/tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.171.61:45551
--webserver_interface=127.10.171.61
--webserver_port=0
--builtin_ntp_servers=127.10.171.20:42033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615
--hive_metastore_uris=thrift://127.0.0.1:35621
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 07:35:55.101671 12781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:35:55.128335 12781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } has no permanent_uuid. Determining permanent_uuid...
W20260926 07:35:55.130848 12731 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.61:45551: connect: Connection refused (error 111)
W20260926 07:35:55.134729 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.61:45551: Network error: Client connection negotiation failed: client connection to 127.10.171.61:45551: connect: Connection refused (error 111)
I20260926 07:35:55.184252 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } attempt: 1
W20260926 07:35:55.187937 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.61:45551: Network error: Client connection negotiation failed: client connection to 127.10.171.61:45551: connect: Connection refused (error 111)
I20260926 07:35:55.256465 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } attempt: 2
W20260926 07:35:55.260221 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.61:45551: Network error: Client connection negotiation failed: client connection to 127.10.171.61:45551: connect: Connection refused (error 111)
I20260926 07:35:55.351660 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } attempt: 3
W20260926 07:35:55.355258 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.61:45551: Network error: Client connection negotiation failed: client connection to 127.10.171.61:45551: connect: Connection refused (error 111)
W20260926 07:35:55.398703 12782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:35:55.399261 12782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:35:55.399648 12782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:35:55.430893 12782 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260926 07:35:55.431213 12782 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260926 07:35:55.431421 12782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:35:55.431603 12782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:35:55.431785 12782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:35:55.431991 12782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:35:55.467645 12782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.171.20:42033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:35621
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.171.61:45551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.171.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:55:20 UTC on ccycloud.quasar-qwhbmo.root.comops.site
build id 4673737
TSAN enabled
I20260926 07:35:55.468917 12782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:35:55.470423 12782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:35:55.487720 12791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:35:55.498706 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } attempt: 4
W20260926 07:35:55.502488 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.61:45551: Network error: Client connection negotiation failed: client connection to 127.10.171.61:45551: connect: Connection refused (error 111)
I20260926 07:35:55.801998 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } attempt: 5
W20260926 07:35:55.806072 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.61:45551: Network error: Client connection negotiation failed: client connection to 127.10.171.61:45551: connect: Connection refused (error 111)
I20260926 07:35:56.353644 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } attempt: 6
W20260926 07:35:56.360628 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.61:45551: Network error: Client connection negotiation failed: client connection to 127.10.171.61:45551: connect: Connection refused (error 111)
W20260926 07:35:55.488396 12794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:35:55.489317 12782 server_base.cc:1034] running on GCE node
W20260926 07:35:55.487766 12792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:35:56.673539 12782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:35:56.676296 12782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:35:56.677659 12782 hybrid_clock.cc:648] HybridClock initialized: now 1790408156677632 us; error 53 us; skew 500 ppm
I20260926 07:35:56.678529 12782 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:35:56.685931 12782 webserver.cc:460] Webserver started at http://127.10.171.61:44225/ using document root <none> and password file <none>
I20260926 07:35:56.686931 12782 fs_manager.cc:359] Metadata directory not provided
I20260926 07:35:56.687161 12782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:35:56.687659 12782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:35:56.692942 12782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/data/instance:
uuid: "7726b3d3f73544318efe4a3be7a63f70"
format_stamp: "Formatted at 2026-09-26 07:35:56 on dist-test-slave-3mmx"
I20260926 07:35:56.694144 12782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/wal/instance:
uuid: "7726b3d3f73544318efe4a3be7a63f70"
format_stamp: "Formatted at 2026-09-26 07:35:56 on dist-test-slave-3mmx"
I20260926 07:35:56.701608 12782 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.003s
I20260926 07:35:56.707352 12804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:35:56.708539 12782 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260926 07:35:56.708868 12782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/data,/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/wal
uuid: "7726b3d3f73544318efe4a3be7a63f70"
format_stamp: "Formatted at 2026-09-26 07:35:56 on dist-test-slave-3mmx"
I20260926 07:35:56.709229 12782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:35:56.746950 12782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:35:56.748709 12782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:35:56.749348 12782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:35:56.819303 12782 rpc_server.cc:307] RPC server started. Bound to: 127.10.171.61:45551
I20260926 07:35:56.819368 12855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.171.61:45551 every 8 connection(s)
I20260926 07:35:56.822046 12782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/data/info.pb
I20260926 07:35:56.824015 10924 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu as pid 12782
I20260926 07:35:56.824772 10924 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/wal/instance
I20260926 07:35:56.827267 12856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:56.827415 10924 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
/tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.171.60:45615
--webserver_interface=127.10.171.60
--webserver_port=0
--builtin_ntp_servers=127.10.171.20:42033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615
--hive_metastore_uris=thrift://127.0.0.1:35621
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 07:35:56.838362 12856 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:35:56.861944 12856 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:35:56.878786 12856 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } has no permanent_uuid. Determining permanent_uuid...
W20260926 07:35:56.880959 12807 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:35:56.884888 12856 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:45615: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:35:56.934387 12856 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } attempt: 1
W20260926 07:35:56.937948 12856 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:45615: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:35:57.006464 12856 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } attempt: 2
W20260926 07:35:57.010213 12856 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:45615: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:35:57.101677 12856 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } attempt: 3
W20260926 07:35:57.105500 12856 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:45615: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:35:57.140790 12857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:35:57.141404 12857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:35:57.141912 12857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:35:57.172258 12857 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260926 07:35:57.172560 12857 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260926 07:35:57.172837 12857 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:35:57.173085 12857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:35:57.173345 12857 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:35:57.173596 12857 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:35:57.208504 12857 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.171.20:42033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:35621
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.171.60:45615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.171.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:55:20 UTC on ccycloud.quasar-qwhbmo.root.comops.site
build id 4673737
TSAN enabled
I20260926 07:35:57.209751 12857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:35:57.211313 12857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:35:57.229581 12868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:35:57.249048 12856 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } attempt: 4
W20260926 07:35:57.253929 12856 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:45615: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:35:57.421239 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } attempt: 7
I20260926 07:35:57.437760 12781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } has no permanent_uuid. Determining permanent_uuid...
W20260926 07:35:57.443717 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:45615: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:35:57.502364 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } attempt: 1
W20260926 07:35:57.506474 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:45615: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:35:57.553454 12856 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } attempt: 5
W20260926 07:35:57.557884 12856 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:45615: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:35:57.588094 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } attempt: 2
W20260926 07:35:57.593304 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:45615: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:35:57.678768 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } attempt: 3
W20260926 07:35:57.683336 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:45615: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:35:57.823752 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } attempt: 4
W20260926 07:35:57.829061 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:45615: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:35:58.105279 12856 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } attempt: 6
W20260926 07:35:58.112779 12856 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:45615: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:35:58.112730 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } attempt: 5
W20260926 07:35:58.116784 12781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.171.60:45615: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:35:57.229595 12870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:35:57.229789 12867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:35:58.431154 12869 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260926 07:35:58.431243 12857 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260926 07:35:58.435025 12857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:35:58.437592 12857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:35:58.439023 12857 hybrid_clock.cc:648] HybridClock initialized: now 1790408158439001 us; error 65 us; skew 500 ppm
I20260926 07:35:58.439862 12857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:35:58.446880 12857 webserver.cc:460] Webserver started at http://127.10.171.60:40625/ using document root <none> and password file <none>
I20260926 07:35:58.447855 12857 fs_manager.cc:359] Metadata directory not provided
I20260926 07:35:58.448096 12857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:35:58.448664 12857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:35:58.453754 12857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/data/instance:
uuid: "0a6ef379bc314e798568d59eba3a3a91"
format_stamp: "Formatted at 2026-09-26 07:35:58 on dist-test-slave-3mmx"
I20260926 07:35:58.454945 12857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/wal/instance:
uuid: "0a6ef379bc314e798568d59eba3a3a91"
format_stamp: "Formatted at 2026-09-26 07:35:58 on dist-test-slave-3mmx"
I20260926 07:35:58.462227 12857 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.007s
I20260926 07:35:58.467783 12882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:35:58.468870 12857 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 07:35:58.469219 12857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/data,/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/wal
uuid: "0a6ef379bc314e798568d59eba3a3a91"
format_stamp: "Formatted at 2026-09-26 07:35:58 on dist-test-slave-3mmx"
I20260926 07:35:58.469570 12857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:35:58.503556 12857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:35:58.505090 12857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:35:58.505528 12857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:35:58.575044 12857 rpc_server.cc:307] RPC server started. Bound to: 127.10.171.60:45615
I20260926 07:35:58.575151 12933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.171.60:45615 every 8 connection(s)
I20260926 07:35:58.578056 12857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/data/info.pb
I20260926 07:35:58.583264 10924 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu as pid 12857
I20260926 07:35:58.583688 10924 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/wal/instance
I20260926 07:35:58.583695 12934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:35:58.602603 12934 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:35:58.629060 12934 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:35:58.640476 12934 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } has no permanent_uuid. Determining permanent_uuid...
I20260926 07:35:58.657449 12934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: Bootstrap starting.
I20260926 07:35:58.663771 12934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:58.665580 12934 log.cc:824] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: Log is configured to *not* fsync() on all Append() calls
I20260926 07:35:58.669119 12781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } attempt: 6
I20260926 07:35:58.670849 12934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: No bootstrap required, opened a new log
I20260926 07:35:58.687170 12781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375: Bootstrap starting.
I20260926 07:35:58.692392 12934 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:35:58.692971 12934 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:58.693245 12934 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a6ef379bc314e798568d59eba3a3a91, State: Initialized, Role: FOLLOWER
I20260926 07:35:58.693392 12781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:58.694043 12934 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:35:58.695191 12781 log.cc:824] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375: Log is configured to *not* fsync() on all Append() calls
I20260926 07:35:58.696923 12942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:35:58.697868 12942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:35:58.698773 12934 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:35:58.702375 12781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375: No bootstrap required, opened a new log
I20260926 07:35:58.738416 12781 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:35:58.739413 12781 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:58.739743 12781 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51342b641d47457d84f48fcbb6c48375, State: Initialized, Role: FOLLOWER
W20260926 07:35:58.740608 12957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:58.740995 12957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:35:58.740763 12781 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:35:58.748998 12960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:35:58.752722 12960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:35:58.752389 12781 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 07:35:58.755247 12957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:58.758216 12957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 07:35:58.760600 12888 tablet.cc:2369] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 07:35:58.767712 12735 tablet.cc:2369] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:35:58.728715 12857 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: OK, instance_id { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" instance_seqno: 1790408155058848 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 07:35:58.770591 12857 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:35:58.772611 12957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:58.772912 12957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 07:35:58.788056 12957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:58.788589 12957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 07:35:58.805133 12957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:58.805696 12957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 07:35:58.822212 12957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:58.824465 12957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 07:35:58.840052 12957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:58.840595 12957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 07:35:58.856077 12957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:58.856792 12957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 07:35:58.871512 12957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:58.872007 12957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:35:58.884547 12960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:35:58.885067 12960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
W20260926 07:35:58.886641 12957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:58.886894 12957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:35:58.888943 12909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "51342b641d47457d84f48fcbb6c48375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a6ef379bc314e798568d59eba3a3a91" is_pre_election: true
I20260926 07:35:58.889843 12909 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51342b641d47457d84f48fcbb6c48375 in term 0.
I20260926 07:35:58.891507 12732 leader_election.cc:302] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375; no voters:
I20260926 07:35:58.893441 12960 leader_election.cc:288] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
I20260926 07:35:58.894065 12960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:35:58.894448 12960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:35:58.894794 12960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 0 FOLLOWER]: Advancing to term 1
W20260926 07:35:58.901086 12957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:58.901397 12957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:35:58.903013 12960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:35:58.908951 12909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "51342b641d47457d84f48fcbb6c48375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a6ef379bc314e798568d59eba3a3a91"
I20260926 07:35:58.909709 12909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:58.922868 12960 leader_election.cc:288] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [CANDIDATE]: Term 1 election: Requested vote from peers 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
I20260926 07:35:58.930512 12909 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51342b641d47457d84f48fcbb6c48375 in term 1.
I20260926 07:35:58.933723 12732 leader_election.cc:302] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375; no voters:
I20260926 07:35:58.935177 12960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:35:58.936803 12960 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 1 LEADER]: Becoming Leader. State: Replica: 51342b641d47457d84f48fcbb6c48375, State: Running, Role: LEADER
W20260926 07:35:58.941732 12957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:58.942037 12957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:35:58.940007 12960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
W20260926 07:35:58.956388 12957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:58.956694 12957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:35:58.977807 12973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51342b641d47457d84f48fcbb6c48375. Latest consensus state: current_term: 1 leader_uuid: "51342b641d47457d84f48fcbb6c48375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:35:58.985651 12973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: This master's current role is: LEADER
W20260926 07:35:59.014065 12957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:59.014395 12957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 07:35:59.024618 12976 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:59.025059 12976 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 07:35:59.029995 12957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:59.030427 12957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:35:59.033145 12978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:35:59.040715 12978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:35:58.990700 12713 master_runner.cc:186] Error getting master registration for 127.10.171.61:45551: OK, instance_id { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" instance_seqno: 1790408156800320 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 07:35:59.050644 12713 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:35:59.050359 12957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:59.051180 12957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 07:35:59.070679 12957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:59.071408 12957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:35:59.074200 12909 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 1 FOLLOWER]: Refusing update from remote peer 51342b641d47457d84f48fcbb6c48375: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:35:59.076480 12973 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 07:35:59.130141 12957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:59.130398 12957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:35:59.129932 12942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51342b641d47457d84f48fcbb6c48375. Latest consensus state: current_term: 1 leader_uuid: "51342b641d47457d84f48fcbb6c48375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:35:59.130719 12942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 07:35:59.145134 12957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:59.145460 12957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:35:59.147065 12960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "51342b641d47457d84f48fcbb6c48375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:35:59.147964 12960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: This master's current role is: LEADER
I20260926 07:35:59.148057 12978 catalog_manager.cc:1347] Generated new cluster ID: 6e2e15d6975946d1a4009a3693fa16a8
I20260926 07:35:59.148383 12978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:35:59.148461 12981 mvcc.cc:205] Tried to move back new op lower bound from 7333511819547303936 to 7333511819055792128. Current Snapshot: MvccSnapshot[applied={T|T < 7333511819547303936}]
I20260926 07:35:59.152980 12942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "51342b641d47457d84f48fcbb6c48375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:35:59.153656 12942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 07:35:59.159406 12957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 07:35:59.159754 12957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:35:59.173179 12856 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } attempt: 7
I20260926 07:35:59.175026 12957 catalog_manager.cc:1259] Loaded cluster ID: 6e2e15d6975946d1a4009a3693fa16a8
I20260926 07:35:59.175295 12957 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: success
W20260926 07:35:59.182226 12957 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260926 07:35:59.182512 12957 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260926 07:35:59.199476 12978 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 07:35:59.203321 12978 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:35:59.218357 12957 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: acquiring CA information for follower catalog manager: success
I20260926 07:35:59.236055 12856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70: Bootstrap starting.
I20260926 07:35:59.243069 12856 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70: Neither blocks nor log segments found. Creating new log.
I20260926 07:35:59.245411 12856 log.cc:824] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70: Log is configured to *not* fsync() on all Append() calls
W20260926 07:35:59.253288 12957 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260926 07:35:59.259904 12856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70: No bootstrap required, opened a new log
W20260926 07:35:59.277565 12957 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260926 07:35:59.280346 12978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375: Generated new TSK 0
I20260926 07:35:59.281476 12978 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:35:59.285813 12978 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260926 07:35:59.290257 12978 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260926 07:35:59.301157 10924 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
/tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.171.1:0
--local_ip_for_outbound_sockets=127.10.171.1
--webserver_interface=127.10.171.1
--webserver_port=0
--tserver_master_addrs=127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615
--builtin_ntp_servers=127.10.171.20:42033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260926 07:35:59.310796 12856 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:35:59.311677 12856 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:35:59.311976 12856 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7726b3d3f73544318efe4a3be7a63f70, State: Initialized, Role: FOLLOWER
I20260926 07:35:59.312959 12856 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:35:59.323603 12856 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:35:59.323729 12831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "51342b641d47457d84f48fcbb6c48375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7726b3d3f73544318efe4a3be7a63f70" is_pre_election: true
I20260926 07:35:59.324581 12831 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51342b641d47457d84f48fcbb6c48375 in term 0.
I20260926 07:35:59.325850 12829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:35:59.330288 12830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "51342b641d47457d84f48fcbb6c48375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7726b3d3f73544318efe4a3be7a63f70"
I20260926 07:35:59.339480 12829 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 1 FOLLOWER]: Refusing update from remote peer 51342b641d47457d84f48fcbb6c48375: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 07:35:59.345458 12973 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:35:59.349440 12987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:35:59.350323 12987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:35:59.354503 12957 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 07:35:59.394438 12995 mvcc.cc:205] Tried to move back new op lower bound from 7333511819547303936 to 7333511819055792128. Current Snapshot: MvccSnapshot[applied={T|T < 7333511819547303936}]
I20260926 07:35:59.419782 12987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51342b641d47457d84f48fcbb6c48375. Latest consensus state: current_term: 1 leader_uuid: "51342b641d47457d84f48fcbb6c48375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:35:59.420538 12987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:35:59.423067 12993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "51342b641d47457d84f48fcbb6c48375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:35:59.423879 12993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:35:59.514194 13008 catalog_manager.cc:1259] Loaded cluster ID: 6e2e15d6975946d1a4009a3693fa16a8
I20260926 07:35:59.514469 13008 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70: loading cluster ID for follower catalog manager: success
I20260926 07:35:59.523661 13008 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70: acquiring CA information for follower catalog manager: success
I20260926 07:35:59.529518 13008 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 07:35:59.858630 12993 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 51342b641d47457d84f48fcbb6c48375)
I20260926 07:35:59.859154 12993 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:35:59.863059 12756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7726b3d3f73544318efe4a3be7a63f70" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "51342b641d47457d84f48fcbb6c48375" is_pre_election: true
I20260926 07:35:59.863544 12993 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
I20260926 07:35:59.865049 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7726b3d3f73544318efe4a3be7a63f70" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a6ef379bc314e798568d59eba3a3a91" is_pre_election: true
I20260926 07:35:59.866914 12807 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:35:59.868948 12993 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:35:59.913851 12973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "51342b641d47457d84f48fcbb6c48375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:35:59.914572 12973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: This master's current role is: LEADER
W20260926 07:35:59.922782 12991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:35:59.923429 12991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:35:59.924577 12991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:35:59.981410 12991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:35:59.983002 12991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.171.1
I20260926 07:36:00.045442 12991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.171.20:42033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.171.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.171.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.171.1
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:55:20 UTC on ccycloud.quasar-qwhbmo.root.comops.site
build id 4673737
TSAN enabled
I20260926 07:36:00.047045 12991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:36:00.049208 12991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 07:36:00.116835 12991 server_base.cc:1034] running on GCE node
I20260926 07:36:00.302558 13023 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 51342b641d47457d84f48fcbb6c48375)
I20260926 07:36:00.304306 13023 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:00.311038 12756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a6ef379bc314e798568d59eba3a3a91" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "51342b641d47457d84f48fcbb6c48375" is_pre_election: true
I20260926 07:36:00.311530 12829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a6ef379bc314e798568d59eba3a3a91" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7726b3d3f73544318efe4a3be7a63f70" is_pre_election: true
I20260926 07:36:00.316738 12886 leader_election.cc:302] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a6ef379bc314e798568d59eba3a3a91; no voters: 51342b641d47457d84f48fcbb6c48375, 7726b3d3f73544318efe4a3be7a63f70
I20260926 07:36:00.319558 13023 leader_election.cc:288] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551)
I20260926 07:36:00.320441 13023 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:36:00.440485 13025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 51342b641d47457d84f48fcbb6c48375)
I20260926 07:36:00.442333 13025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:00.447813 12756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7726b3d3f73544318efe4a3be7a63f70" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "51342b641d47457d84f48fcbb6c48375" is_pre_election: true
I20260926 07:36:00.450784 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7726b3d3f73544318efe4a3be7a63f70" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a6ef379bc314e798568d59eba3a3a91" is_pre_election: true
I20260926 07:36:00.452302 12807 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:00.458565 13025 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
I20260926 07:36:00.460530 13025 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:36:01.488945 13027 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 51342b641d47457d84f48fcbb6c48375)
I20260926 07:36:01.489744 13027 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:01.492702 12756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7726b3d3f73544318efe4a3be7a63f70" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "51342b641d47457d84f48fcbb6c48375" is_pre_election: true
I20260926 07:36:01.494791 13027 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
I20260926 07:36:01.495786 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7726b3d3f73544318efe4a3be7a63f70" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a6ef379bc314e798568d59eba3a3a91" is_pre_election: true
I20260926 07:36:01.497298 12807 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:01.498008 13027 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260926 07:36:00.124567 13020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:36:01.120692 12991 thread.cc:536] Waited for 1000ms trying to join with AWS (tid 13017)
W20260926 07:36:00.117836 13017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:36:00.117686 13018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:36:01.815521 12991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:36:01.825542 12991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:36:01.826982 12991 hybrid_clock.cc:648] HybridClock initialized: now 1790408161826946 us; error 55 us; skew 500 ppm
I20260926 07:36:01.828202 12991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:36:01.859459 12991 webserver.cc:460] Webserver started at http://127.10.171.1:44687/ using document root <none> and password file <none>
I20260926 07:36:01.861129 12991 fs_manager.cc:359] Metadata directory not provided
I20260926 07:36:01.861486 12991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:36:01.862143 12991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:36:01.869800 12991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/data/instance:
uuid: "095560781b2d4e58b3804fce75312b90"
format_stamp: "Formatted at 2026-09-26 07:36:01 on dist-test-slave-3mmx"
I20260926 07:36:01.871583 12991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/wal/instance:
uuid: "095560781b2d4e58b3804fce75312b90"
format_stamp: "Formatted at 2026-09-26 07:36:01 on dist-test-slave-3mmx"
I20260926 07:36:01.888363 12991 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.006s sys 0.008s
I20260926 07:36:01.902570 13032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:36:01.904031 12991 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.002s sys 0.007s
I20260926 07:36:01.904577 12991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/wal
uuid: "095560781b2d4e58b3804fce75312b90"
format_stamp: "Formatted at 2026-09-26 07:36:01 on dist-test-slave-3mmx"
I20260926 07:36:01.905215 12991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:36:01.992446 12991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:36:01.994709 12991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:36:01.995401 12991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:36:01.998998 12991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:36:02.005283 12991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:36:02.005656 12991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260926 07:36:02.006073 12991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:36:02.006354 12991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:36:02.028793 13041 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 51342b641d47457d84f48fcbb6c48375)
I20260926 07:36:02.030129 13041 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:02.033063 12756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a6ef379bc314e798568d59eba3a3a91" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "51342b641d47457d84f48fcbb6c48375" is_pre_election: true
I20260926 07:36:02.036490 12829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a6ef379bc314e798568d59eba3a3a91" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7726b3d3f73544318efe4a3be7a63f70" is_pre_election: true
I20260926 07:36:02.037997 12884 leader_election.cc:302] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a6ef379bc314e798568d59eba3a3a91; no voters: 51342b641d47457d84f48fcbb6c48375, 7726b3d3f73544318efe4a3be7a63f70
I20260926 07:36:02.043557 13041 leader_election.cc:288] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551)
I20260926 07:36:02.045645 13041 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:36:02.067250 13058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 51342b641d47457d84f48fcbb6c48375)
I20260926 07:36:02.067817 13058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:02.071760 12756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7726b3d3f73544318efe4a3be7a63f70" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "51342b641d47457d84f48fcbb6c48375" is_pre_election: true
I20260926 07:36:02.075517 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7726b3d3f73544318efe4a3be7a63f70" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a6ef379bc314e798568d59eba3a3a91" is_pre_election: true
I20260926 07:36:02.076378 12908 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7726b3d3f73544318efe4a3be7a63f70 in term 1.
I20260926 07:36:02.078531 12807 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a6ef379bc314e798568d59eba3a3a91, 7726b3d3f73544318efe4a3be7a63f70; no voters: 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:02.122457 13058 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
I20260926 07:36:02.123517 13058 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 07:36:02.125161 13058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 51342b641d47457d84f48fcbb6c48375)
I20260926 07:36:02.129547 13058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 1 FOLLOWER]: Advancing to term 2
W20260926 07:36:02.146744 12807 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.60:45615, user_credentials={real_user=slave}} blocked reactor thread for 68333.4us
I20260926 07:36:02.158201 13058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:02.161175 12829 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 2 FOLLOWER]: Rejecting Update request from peer 51342b641d47457d84f48fcbb6c48375 for earlier term 1. Current term is 2. Ops: []
I20260926 07:36:02.166045 13061 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260926 07:36:02.167140 13061 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 1 LEADER]: Stepping down as leader of term 1
I20260926 07:36:02.167550 13061 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 51342b641d47457d84f48fcbb6c48375, State: Running, Role: LEADER
I20260926 07:36:02.168428 13061 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:02.172004 12756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7726b3d3f73544318efe4a3be7a63f70" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "51342b641d47457d84f48fcbb6c48375"
I20260926 07:36:02.174360 13061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 1 FOLLOWER]: Advancing to term 2
I20260926 07:36:02.177345 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7726b3d3f73544318efe4a3be7a63f70" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a6ef379bc314e798568d59eba3a3a91"
I20260926 07:36:02.181016 13058 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 2 election: Requested vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
I20260926 07:36:02.211107 12756 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7726b3d3f73544318efe4a3be7a63f70 in term 2.
I20260926 07:36:02.219435 12976 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 07:36:02.222856 12808 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51342b641d47457d84f48fcbb6c48375, 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91
I20260926 07:36:02.224076 13058 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 07:36:02.224813 13058 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 2 LEADER]: Becoming Leader. State: Replica: 7726b3d3f73544318efe4a3be7a63f70, State: Running, Role: LEADER
I20260926 07:36:02.225889 13058 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:02.237028 13062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7726b3d3f73544318efe4a3be7a63f70. Latest consensus state: current_term: 2 leader_uuid: "7726b3d3f73544318efe4a3be7a63f70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:02.237815 13062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: This master's current role is: LEADER
I20260926 07:36:02.248417 13082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:36:02.266129 13082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:36:02.271281 13082 catalog_manager.cc:1259] Loaded cluster ID: 6e2e15d6975946d1a4009a3693fa16a8
I20260926 07:36:02.273017 13082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:36:02.279322 13082 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:36:02.284642 13082 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70: Loaded TSK: 0
I20260926 07:36:02.287245 13082 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:36:02.292677 13082 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260926 07:36:02.299402 13082 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260926 07:36:02.409617 12991 rpc_server.cc:307] RPC server started. Bound to: 127.10.171.1:37253
I20260926 07:36:02.409770 13152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.171.1:37253 every 8 connection(s)
I20260926 07:36:02.413199 12991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/data/info.pb
I20260926 07:36:02.422585 10924 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu as pid 12991
I20260926 07:36:02.423089 10924 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/wal/instance
I20260926 07:36:02.436908 10924 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
/tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.171.2:0
--local_ip_for_outbound_sockets=127.10.171.2
--webserver_interface=127.10.171.2
--webserver_port=0
--tserver_master_addrs=127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615
--builtin_ntp_servers=127.10.171.20:42033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260926 07:36:02.529083 13158 heartbeater.cc:347] Connected to a master server at 127.10.171.62:36693
I20260926 07:36:02.529572 13158 heartbeater.cc:464] Registering TS with master...
I20260926 07:36:02.530790 13158 heartbeater.cc:511] Master 127.10.171.62:36693 requested a full tablet report, sending...
I20260926 07:36:02.533316 13153 heartbeater.cc:347] Connected to a master server at 127.10.171.61:45551
I20260926 07:36:02.533643 13153 heartbeater.cc:464] Registering TS with master...
I20260926 07:36:02.534412 13153 heartbeater.cc:511] Master 127.10.171.61:45551 requested a full tablet report, sending...
I20260926 07:36:02.534657 12746 ts_manager.cc:194] Registered new tserver with Master: 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253)
I20260926 07:36:02.536865 12821 ts_manager.cc:194] Registered new tserver with Master: 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253)
I20260926 07:36:02.539938 12821 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.171.1:54351
I20260926 07:36:02.544962 13154 heartbeater.cc:347] Connected to a master server at 127.10.171.60:45615
I20260926 07:36:02.545312 13154 heartbeater.cc:464] Registering TS with master...
I20260926 07:36:02.546028 13154 heartbeater.cc:511] Master 127.10.171.60:45615 requested a full tablet report, sending...
I20260926 07:36:02.548813 12899 ts_manager.cc:194] Registered new tserver with Master: 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253)
I20260926 07:36:02.673547 13164 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 51342b641d47457d84f48fcbb6c48375)
I20260926 07:36:02.674255 13164 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:02.679189 12756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a6ef379bc314e798568d59eba3a3a91" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "51342b641d47457d84f48fcbb6c48375" is_pre_election: true
I20260926 07:36:02.680019 12756 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a6ef379bc314e798568d59eba3a3a91 in current term 2: Already voted for candidate 7726b3d3f73544318efe4a3be7a63f70 in this term.
I20260926 07:36:02.680174 13164 leader_election.cc:288] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551)
I20260926 07:36:02.680801 12829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a6ef379bc314e798568d59eba3a3a91" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7726b3d3f73544318efe4a3be7a63f70" is_pre_election: true
I20260926 07:36:02.682296 12884 leader_election.cc:302] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a6ef379bc314e798568d59eba3a3a91; no voters: 51342b641d47457d84f48fcbb6c48375, 7726b3d3f73544318efe4a3be7a63f70
I20260926 07:36:02.683403 13164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 1 FOLLOWER]: Advancing to term 2
I20260926 07:36:02.692220 13164 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 07:36:02.725001 12756 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 2 FOLLOWER]: Refusing update from remote peer 7726b3d3f73544318efe4a3be7a63f70: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260926 07:36:02.726703 13062 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260926 07:36:02.738479 12908 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 2 FOLLOWER]: Refusing update from remote peer 7726b3d3f73544318efe4a3be7a63f70: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260926 07:36:02.740096 13062 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260926 07:36:02.740285 13061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7726b3d3f73544318efe4a3be7a63f70. Latest consensus state: current_term: 2 leader_uuid: "7726b3d3f73544318efe4a3be7a63f70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:02.741083 13061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:36:02.761371 13164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7726b3d3f73544318efe4a3be7a63f70. Latest consensus state: current_term: 2 leader_uuid: "7726b3d3f73544318efe4a3be7a63f70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:02.762174 13164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:36:02.771075 13061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "7726b3d3f73544318efe4a3be7a63f70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:02.771803 13061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:36:02.782763 13167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "7726b3d3f73544318efe4a3be7a63f70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:02.783807 13167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: This master's current role is: LEADER
I20260926 07:36:02.785066 13167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "7726b3d3f73544318efe4a3be7a63f70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:02.785812 13167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: This master's current role is: LEADER
I20260926 07:36:02.786451 13164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "7726b3d3f73544318efe4a3be7a63f70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:02.787192 13164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 07:36:02.930522 13157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:36:02.931080 13157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:36:02.931670 13157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:36:02.963974 13157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:36:02.965018 13157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.171.2
I20260926 07:36:03.000167 13157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.171.20:42033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.171.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.171.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.171.2
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:55:20 UTC on ccycloud.quasar-qwhbmo.root.comops.site
build id 4673737
TSAN enabled
I20260926 07:36:03.001281 13157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:36:03.002982 13157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:36:03.022239 13178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:36:03.045167 13153 heartbeater.cc:503] Master 127.10.171.61:45551 was elected leader, sending a full tablet report...
I20260926 07:36:03.256738 13061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7726b3d3f73544318efe4a3be7a63f70)
I20260926 07:36:03.257486 13061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:03.260293 13061 leader_election.cc:288] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
I20260926 07:36:03.260869 12829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "51342b641d47457d84f48fcbb6c48375" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7726b3d3f73544318efe4a3be7a63f70" is_pre_election: true
I20260926 07:36:03.261477 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "51342b641d47457d84f48fcbb6c48375" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a6ef379bc314e798568d59eba3a3a91" is_pre_election: true
I20260926 07:36:03.263403 12732 leader_election.cc:302] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51342b641d47457d84f48fcbb6c48375; no voters: 0a6ef379bc314e798568d59eba3a3a91, 7726b3d3f73544318efe4a3be7a63f70
I20260926 07:36:03.264461 13061 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260926 07:36:03.687063 13184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7726b3d3f73544318efe4a3be7a63f70)
I20260926 07:36:03.687583 13184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:03.689419 13184 leader_election.cc:288] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551)
I20260926 07:36:03.690598 12829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a6ef379bc314e798568d59eba3a3a91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7726b3d3f73544318efe4a3be7a63f70" is_pre_election: true
I20260926 07:36:03.691969 12756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a6ef379bc314e798568d59eba3a3a91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "51342b641d47457d84f48fcbb6c48375" is_pre_election: true
I20260926 07:36:03.694787 12886 leader_election.cc:302] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a6ef379bc314e798568d59eba3a3a91; no voters: 51342b641d47457d84f48fcbb6c48375, 7726b3d3f73544318efe4a3be7a63f70
I20260926 07:36:03.695776 13184 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260926 07:36:03.022886 13177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:36:03.023443 13157 server_base.cc:1034] running on GCE node
W20260926 07:36:03.023077 13180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:36:04.238989 13157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:36:04.241233 13157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:36:04.242571 13157 hybrid_clock.cc:648] HybridClock initialized: now 1790408164242545 us; error 49 us; skew 500 ppm
I20260926 07:36:04.243391 13157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:36:04.248905 13187 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7726b3d3f73544318efe4a3be7a63f70)
I20260926 07:36:04.249500 13187 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:04.250146 13157 webserver.cc:460] Webserver started at http://127.10.171.2:43699/ using document root <none> and password file <none>
I20260926 07:36:04.251462 13157 fs_manager.cc:359] Metadata directory not provided
I20260926 07:36:04.251732 13157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:36:04.252363 13157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:36:04.252296 13187 leader_election.cc:288] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
I20260926 07:36:04.253981 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "51342b641d47457d84f48fcbb6c48375" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a6ef379bc314e798568d59eba3a3a91" is_pre_election: true
I20260926 07:36:04.254571 12829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "51342b641d47457d84f48fcbb6c48375" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7726b3d3f73544318efe4a3be7a63f70" is_pre_election: true
I20260926 07:36:04.255901 12731 leader_election.cc:302] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51342b641d47457d84f48fcbb6c48375; no voters: 0a6ef379bc314e798568d59eba3a3a91, 7726b3d3f73544318efe4a3be7a63f70
I20260926 07:36:04.256731 13187 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260926 07:36:04.259282 13157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/data/instance:
uuid: "16209ececd2d420588b390f95db9b92a"
format_stamp: "Formatted at 2026-09-26 07:36:04 on dist-test-slave-3mmx"
I20260926 07:36:04.260449 13157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/wal/instance:
uuid: "16209ececd2d420588b390f95db9b92a"
format_stamp: "Formatted at 2026-09-26 07:36:04 on dist-test-slave-3mmx"
I20260926 07:36:04.268076 13157 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260926 07:36:04.273743 13190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:36:04.274856 13157 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260926 07:36:04.275202 13157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/wal
uuid: "16209ececd2d420588b390f95db9b92a"
format_stamp: "Formatted at 2026-09-26 07:36:04 on dist-test-slave-3mmx"
I20260926 07:36:04.275548 13157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:36:04.289162 13191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7726b3d3f73544318efe4a3be7a63f70)
I20260926 07:36:04.289541 13191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:04.291729 13191 leader_election.cc:288] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551)
I20260926 07:36:04.292665 12756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a6ef379bc314e798568d59eba3a3a91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "51342b641d47457d84f48fcbb6c48375" is_pre_election: true
I20260926 07:36:04.292759 12829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a6ef379bc314e798568d59eba3a3a91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7726b3d3f73544318efe4a3be7a63f70" is_pre_election: true
I20260926 07:36:04.293134 12756 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a6ef379bc314e798568d59eba3a3a91 in term 2.
I20260926 07:36:04.294046 12886 leader_election.cc:302] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375; no voters:
I20260926 07:36:04.294795 13191 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260926 07:36:04.295141 13191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7726b3d3f73544318efe4a3be7a63f70)
I20260926 07:36:04.295511 13191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 2 FOLLOWER]: Advancing to term 3
I20260926 07:36:04.303247 13191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:04.305543 13191 leader_election.cc:288] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 3 election: Requested vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551)
I20260926 07:36:04.306479 12756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a6ef379bc314e798568d59eba3a3a91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "51342b641d47457d84f48fcbb6c48375"
I20260926 07:36:04.306980 12756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 2 FOLLOWER]: Advancing to term 3
I20260926 07:36:04.307188 12829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a6ef379bc314e798568d59eba3a3a91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "7726b3d3f73544318efe4a3be7a63f70"
I20260926 07:36:04.313964 12756 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a6ef379bc314e798568d59eba3a3a91 in term 3.
I20260926 07:36:04.315012 12886 leader_election.cc:302] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375; no voters: 7726b3d3f73544318efe4a3be7a63f70
I20260926 07:36:04.315825 13191 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 07:36:04.317631 13191 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 3 LEADER]: Becoming Leader. State: Replica: 0a6ef379bc314e798568d59eba3a3a91, State: Running, Role: LEADER
I20260926 07:36:04.318411 13191 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:04.321959 13196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a6ef379bc314e798568d59eba3a3a91. Latest consensus state: current_term: 3 leader_uuid: "0a6ef379bc314e798568d59eba3a3a91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:04.322741 13196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: This master's current role is: LEADER
I20260926 07:36:04.324308 13199 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:36:04.331140 13199 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:36:04.331435 12908 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 3 LEADER]: Rejecting Update request from peer 7726b3d3f73544318efe4a3be7a63f70 for earlier term 2. Current term is 3. Ops: []
I20260926 07:36:04.332880 13198 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 07:36:04.334136 13198 consensus_queue.cc:1225] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [LEADER]: Peer 0a6ef379bc314e798568d59eba3a3a91 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260926 07:36:04.334570 13200 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 2 LEADER]: Stepping down as leader of term 2
I20260926 07:36:04.334794 13199 catalog_manager.cc:1259] Loaded cluster ID: 6e2e15d6975946d1a4009a3693fa16a8
I20260926 07:36:04.334865 13200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7726b3d3f73544318efe4a3be7a63f70, State: Running, Role: LEADER
I20260926 07:36:04.335075 13199 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:36:04.335564 13200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:04.336735 13157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:36:04.336805 13200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 2 FOLLOWER]: Advancing to term 3
I20260926 07:36:04.338713 13199 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:36:04.338690 13157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:36:04.339247 13157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:36:04.341483 13199 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: Loaded TSK: 0
I20260926 07:36:04.342732 13157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:36:04.343200 13199 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:36:04.346261 13199 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260926 07:36:04.347293 13157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:36:04.347474 13157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:36:04.347859 13157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:36:04.348001 13157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:36:04.349952 13199 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260926 07:36:04.496553 13157 rpc_server.cc:307] RPC server started. Bound to: 127.10.171.2:39565
I20260926 07:36:04.496627 13308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.171.2:39565 every 8 connection(s)
I20260926 07:36:04.499269 13157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/data/info.pb
I20260926 07:36:04.502002 10924 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu as pid 13157
I20260926 07:36:04.502563 10924 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/wal/instance
I20260926 07:36:04.514680 10924 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
/tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.171.3:0
--local_ip_for_outbound_sockets=127.10.171.3
--webserver_interface=127.10.171.3
--webserver_port=0
--tserver_master_addrs=127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615
--builtin_ntp_servers=127.10.171.20:42033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260926 07:36:04.579741 13311 heartbeater.cc:347] Connected to a master server at 127.10.171.62:36693
I20260926 07:36:04.580238 13311 heartbeater.cc:464] Registering TS with master...
I20260926 07:36:04.580762 13310 heartbeater.cc:347] Connected to a master server at 127.10.171.60:45615
I20260926 07:36:04.581178 13310 heartbeater.cc:464] Registering TS with master...
I20260926 07:36:04.581673 13311 heartbeater.cc:511] Master 127.10.171.62:36693 requested a full tablet report, sending...
I20260926 07:36:04.582038 13310 heartbeater.cc:511] Master 127.10.171.60:45615 requested a full tablet report, sending...
I20260926 07:36:04.584517 12899 ts_manager.cc:194] Registered new tserver with Master: 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
I20260926 07:36:04.585521 12746 ts_manager.cc:194] Registered new tserver with Master: 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
I20260926 07:36:04.587481 12899 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.171.2:51303
I20260926 07:36:04.590541 13309 heartbeater.cc:347] Connected to a master server at 127.10.171.61:45551
I20260926 07:36:04.590934 13309 heartbeater.cc:464] Registering TS with master...
I20260926 07:36:04.591645 13309 heartbeater.cc:511] Master 127.10.171.61:45551 requested a full tablet report, sending...
I20260926 07:36:04.594722 12821 ts_manager.cc:194] Registered new tserver with Master: 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
I20260926 07:36:04.806871 12756 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 3 FOLLOWER]: Refusing update from remote peer 0a6ef379bc314e798568d59eba3a3a91: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260926 07:36:04.808303 13196 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 07:36:04.813378 13320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a6ef379bc314e798568d59eba3a3a91. Latest consensus state: current_term: 3 leader_uuid: "0a6ef379bc314e798568d59eba3a3a91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:04.814244 13320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:36:04.818061 13191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "0a6ef379bc314e798568d59eba3a3a91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:04.818919 13191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: This master's current role is: LEADER
I20260926 07:36:04.822595 12829 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 3 FOLLOWER]: Refusing update from remote peer 0a6ef379bc314e798568d59eba3a3a91: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260926 07:36:04.824055 13191 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 07:36:04.837699 13320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "0a6ef379bc314e798568d59eba3a3a91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:04.838398 13320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:36:04.840935 13327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "0a6ef379bc314e798568d59eba3a3a91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:04.841790 13327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:36:04.841135 13200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a6ef379bc314e798568d59eba3a3a91. Latest consensus state: current_term: 3 leader_uuid: "0a6ef379bc314e798568d59eba3a3a91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:04.842638 13200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 07:36:04.915419 13319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:36:04.915910 13319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:36:04.916466 13319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:36:04.948294 13319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:36:04.949254 13319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.171.3
I20260926 07:36:04.984205 13319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.171.20:42033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.171.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.171.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.171.3
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:55:20 UTC on ccycloud.quasar-qwhbmo.root.comops.site
build id 4673737
TSAN enabled
I20260926 07:36:04.985446 13319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:36:04.987243 13319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:36:05.007952 13333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:36:05.064054 13154 heartbeater.cc:503] Master 127.10.171.60:45615 was elected leader, sending a full tablet report...
I20260926 07:36:05.092568 13310 heartbeater.cc:503] Master 127.10.171.60:45615 was elected leader, sending a full tablet report...
I20260926 07:36:05.263070 13191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "0a6ef379bc314e798568d59eba3a3a91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:05.263844 13191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: This master's current role is: LEADER
I20260926 07:36:05.318117 13320 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:36:05.318544 13320 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:05.320086 13320 leader_election.cc:288] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
I20260926 07:36:05.321125 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "51342b641d47457d84f48fcbb6c48375" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0a6ef379bc314e798568d59eba3a3a91" is_pre_election: true
I20260926 07:36:05.321350 12829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "51342b641d47457d84f48fcbb6c48375" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7726b3d3f73544318efe4a3be7a63f70" is_pre_election: true
I20260926 07:36:05.322710 12731 leader_election.cc:302] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51342b641d47457d84f48fcbb6c48375; no voters: 0a6ef379bc314e798568d59eba3a3a91, 7726b3d3f73544318efe4a3be7a63f70
I20260926 07:36:05.323330 13320 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260926 07:36:05.767409 13340 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:36:05.768183 13340 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:05.770639 13340 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
I20260926 07:36:05.771468 12756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7726b3d3f73544318efe4a3be7a63f70" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "51342b641d47457d84f48fcbb6c48375" is_pre_election: true
I20260926 07:36:05.772403 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7726b3d3f73544318efe4a3be7a63f70" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0a6ef379bc314e798568d59eba3a3a91" is_pre_election: true
I20260926 07:36:05.773824 12807 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:05.774461 13340 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260926 07:36:05.008332 13336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:36:05.008833 13334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:36:05.009347 13319 server_base.cc:1034] running on GCE node
I20260926 07:36:06.233837 13319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:36:06.236266 13319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:36:06.237628 13319 hybrid_clock.cc:648] HybridClock initialized: now 1790408166237605 us; error 50 us; skew 500 ppm
I20260926 07:36:06.238405 13319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:36:06.244807 13319 webserver.cc:460] Webserver started at http://127.10.171.3:39939/ using document root <none> and password file <none>
I20260926 07:36:06.245841 13319 fs_manager.cc:359] Metadata directory not provided
I20260926 07:36:06.246062 13319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:36:06.246537 13319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 07:36:06.251605 13319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/data/instance:
uuid: "a6ff3db459444683972a22263d29009b"
format_stamp: "Formatted at 2026-09-26 07:36:06 on dist-test-slave-3mmx"
I20260926 07:36:06.252998 13319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/wal/instance:
uuid: "a6ff3db459444683972a22263d29009b"
format_stamp: "Formatted at 2026-09-26 07:36:06 on dist-test-slave-3mmx"
I20260926 07:36:06.261790 13319 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.008s sys 0.000s
I20260926 07:36:06.267989 13345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:36:06.269289 13319 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 07:36:06.269577 13319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/wal
uuid: "a6ff3db459444683972a22263d29009b"
format_stamp: "Formatted at 2026-09-26 07:36:06 on dist-test-slave-3mmx"
I20260926 07:36:06.269870 13319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:36:06.307264 13319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:36:06.308902 13319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:36:06.309372 13319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:36:06.312214 13319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 07:36:06.316416 13319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 07:36:06.316619 13319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 07:36:06.316826 13319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 07:36:06.316956 13319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 07:36:06.319718 13353 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:36:06.320163 13353 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:06.321933 13353 leader_election.cc:288] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
I20260926 07:36:06.322837 12829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "51342b641d47457d84f48fcbb6c48375" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7726b3d3f73544318efe4a3be7a63f70" is_pre_election: true
I20260926 07:36:06.323130 12908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "51342b641d47457d84f48fcbb6c48375" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0a6ef379bc314e798568d59eba3a3a91" is_pre_election: true
I20260926 07:36:06.324716 12732 leader_election.cc:302] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51342b641d47457d84f48fcbb6c48375; no voters: 0a6ef379bc314e798568d59eba3a3a91, 7726b3d3f73544318efe4a3be7a63f70
I20260926 07:36:06.325596 13353 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260926 07:36:06.469079 13319 rpc_server.cc:307] RPC server started. Bound to: 127.10.171.3:34663
I20260926 07:36:06.469184 13458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.171.3:34663 every 8 connection(s)
I20260926 07:36:06.472236 13319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/data/info.pb
I20260926 07:36:06.477238 10924 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu as pid 13319
I20260926 07:36:06.477839 10924 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/wal/instance
I20260926 07:36:06.530507 13459 heartbeater.cc:347] Connected to a master server at 127.10.171.61:45551
I20260926 07:36:06.531064 13459 heartbeater.cc:464] Registering TS with master...
I20260926 07:36:06.531219 13460 heartbeater.cc:347] Connected to a master server at 127.10.171.60:45615
I20260926 07:36:06.531556 13460 heartbeater.cc:464] Registering TS with master...
I20260926 07:36:06.532660 13459 heartbeater.cc:511] Master 127.10.171.61:45551 requested a full tablet report, sending...
I20260926 07:36:06.534087 13460 heartbeater.cc:511] Master 127.10.171.60:45615 requested a full tablet report, sending...
I20260926 07:36:06.535905 12821 ts_manager.cc:194] Registered new tserver with Master: a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
I20260926 07:36:06.536293 12899 ts_manager.cc:194] Registered new tserver with Master: a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
I20260926 07:36:06.538219 12899 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.171.3:43709
I20260926 07:36:06.538846 13461 heartbeater.cc:347] Connected to a master server at 127.10.171.62:36693
I20260926 07:36:06.539187 13461 heartbeater.cc:464] Registering TS with master...
I20260926 07:36:06.540158 13461 heartbeater.cc:511] Master 127.10.171.62:36693 requested a full tablet report, sending...
I20260926 07:36:06.542330 12745 ts_manager.cc:194] Registered new tserver with Master: a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
I20260926 07:36:06.561685 10924 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 07:36:06.622001 12899 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52138:
name: "default.Table_d7f4c6a111734e628f4183118d11aaef"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:36:06.622025 12898 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52138:
name: "default.Table_5280e066b7404013a7c7bdfe8e9bc18b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:36:06.625299 12897 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52138:
name: "default.Table_b9246fcaf5ed4103b8c7835af689efab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:36:06.630625 12896 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52138:
name: "default.Table_708d07ef404b42629d7c958b81980ca1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:36:06.636305 12899 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_d7f4c6a111734e628f4183118d11aaef in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:36:06.647047 12897 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_b9246fcaf5ed4103b8c7835af689efab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:36:06.652134 12898 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_5280e066b7404013a7c7bdfe8e9bc18b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:36:06.656208 12896 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_708d07ef404b42629d7c958b81980ca1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:36:06.664884 10924 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu with pid 12857
W20260926 07:36:06.706300 13472 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.171.60:45615)
I20260926 07:36:06.763634 13494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:36:06.764402 13494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:06.766502 13494 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
I20260926 07:36:06.767787 12756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7726b3d3f73544318efe4a3be7a63f70" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "51342b641d47457d84f48fcbb6c48375" is_pre_election: true
W20260926 07:36:06.769918 12807 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111) [suppressed 6 similar messages]
W20260926 07:36:06.778903 12807 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:36:06.779386 12807 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:06.781356 13494 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260926 07:36:06.898464 13497 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:36:06.899026 13497 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:06.903728 12829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "51342b641d47457d84f48fcbb6c48375" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7726b3d3f73544318efe4a3be7a63f70" is_pre_election: true
I20260926 07:36:06.904326 12829 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51342b641d47457d84f48fcbb6c48375 in term 3.
I20260926 07:36:06.905448 12731 leader_election.cc:302] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51342b641d47457d84f48fcbb6c48375, 7726b3d3f73544318efe4a3be7a63f70; no voters:
I20260926 07:36:06.912581 13497 leader_election.cc:288] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
I20260926 07:36:06.913515 13497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260926 07:36:06.916213 13497 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:36:06.916980 13497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 3 FOLLOWER]: Advancing to term 4
W20260926 07:36:06.923985 12732 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111) [suppressed 12 similar messages]
W20260926 07:36:06.931028 12732 leader_election.cc:334] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:36:06.931229 13497 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:06.934029 13497 leader_election.cc:288] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [CANDIDATE]: Term 4 election: Requested vote from peers 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
I20260926 07:36:06.934686 12829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "51342b641d47457d84f48fcbb6c48375" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "7726b3d3f73544318efe4a3be7a63f70"
I20260926 07:36:06.935189 12829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 3 FOLLOWER]: Advancing to term 4
W20260926 07:36:06.939634 12732 leader_election.cc:334] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:36:06.941699 12829 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51342b641d47457d84f48fcbb6c48375 in term 4.
I20260926 07:36:06.942703 12731 leader_election.cc:302] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 51342b641d47457d84f48fcbb6c48375, 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91
I20260926 07:36:06.943421 13497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 4 FOLLOWER]: Leader election won for term 4
I20260926 07:36:06.943848 13497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [term 4 LEADER]: Becoming Leader. State: Replica: 51342b641d47457d84f48fcbb6c48375, State: Running, Role: LEADER
I20260926 07:36:06.944554 13497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:06.948458 13499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51342b641d47457d84f48fcbb6c48375. Latest consensus state: current_term: 4 leader_uuid: "51342b641d47457d84f48fcbb6c48375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:06.949242 13499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: This master's current role is: LEADER
I20260926 07:36:06.950834 13502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:36:06.962555 13502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:36:06.966563 13502 catalog_manager.cc:1259] Loaded cluster ID: 6e2e15d6975946d1a4009a3693fa16a8
I20260926 07:36:06.966831 13502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:36:06.971802 13502 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:36:06.975616 13502 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375: Loaded TSK: 0
I20260926 07:36:06.978753 13502 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:36:06.982547 13502 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260926 07:36:06.988495 13502 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260926 07:36:07.014955 12745 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45538:
name: "default.Table_708d07ef404b42629d7c958b81980ca1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:36:07.017354 12746 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45538:
name: "default.Table_b9246fcaf5ed4103b8c7835af689efab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:36:07.017745 12744 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45538:
name: "default.Table_5280e066b7404013a7c7bdfe8e9bc18b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:36:07.019429 12743 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45538:
name: "default.Table_d7f4c6a111734e628f4183118d11aaef"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:36:07.027827 12745 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_708d07ef404b42629d7c958b81980ca1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:36:07.031047 12746 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_b9246fcaf5ed4103b8c7835af689efab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:36:07.033314 12743 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_d7f4c6a111734e628f4183118d11aaef in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 07:36:07.035267 12744 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_5280e066b7404013a7c7bdfe8e9bc18b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 07:36:07.042483 13460 heartbeater.cc:503] Master 127.10.171.60:45615 was elected leader, sending a full tablet report...
W20260926 07:36:07.055291 13460 heartbeater.cc:644] Failed to heartbeat to 127.10.171.60:45615 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:36:07.101904 13154 heartbeater.cc:644] Failed to heartbeat to 127.10.171.60:45615 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:36:07.119762 13310 heartbeater.cc:644] Failed to heartbeat to 127.10.171.60:45615 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:36:07.242143 12745 catalog_manager.cc:2148] created HMS catalog entry for table [id=762c449a1c174102af79fc5e64f6502f]
I20260926 07:36:07.267232 12829 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 4 FOLLOWER]: Refusing update from remote peer 51342b641d47457d84f48fcbb6c48375: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 8. (index mismatch)
I20260926 07:36:07.268774 13499 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260926 07:36:07.269425 12732 consensus_peers.cc:487] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 -> Peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Couldn't send request to peer 0a6ef379bc314e798568d59eba3a3a91. Status: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 07:36:07.292255 13494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51342b641d47457d84f48fcbb6c48375. Latest consensus state: current_term: 4 leader_uuid: "51342b641d47457d84f48fcbb6c48375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:07.293165 13494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:36:07.297969 13506 mvcc.cc:205] Tried to move back new op lower bound from 7333511853093654528 to 7333511851811454976. Current Snapshot: MvccSnapshot[applied={T|T < 7333511853093654528}]
I20260926 07:36:07.303010 13494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "51342b641d47457d84f48fcbb6c48375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:07.303696 13494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:36:07.309794 13497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "51342b641d47457d84f48fcbb6c48375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:07.310570 13497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375 [sys.catalog]: This master's current role is: LEADER
I20260926 07:36:07.323462 12745 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45538:
table { table_name: "default.Table_708d07ef404b42629d7c958b81980ca1" } new_table_name: "default.Table_9fbcc2f37250434e80c28330fb80f80b" modify_external_catalogs: true
I20260926 07:36:07.384526 13088 tablet_service.cc:1467] Processing CreateTablet for tablet f53c6d2017c143e7bb396862fbe7a957 (DEFAULT_TABLE table=default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:07.386408 13088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f53c6d2017c143e7bb396862fbe7a957. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:07.385831 13244 tablet_service.cc:1467] Processing CreateTablet for tablet f53c6d2017c143e7bb396862fbe7a957 (DEFAULT_TABLE table=default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:07.387522 13244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f53c6d2017c143e7bb396862fbe7a957. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:07.393234 13394 tablet_service.cc:1467] Processing CreateTablet for tablet f53c6d2017c143e7bb396862fbe7a957 (DEFAULT_TABLE table=default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:07.395048 13394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f53c6d2017c143e7bb396862fbe7a957. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:07.427013 13519 tablet_bootstrap.cc:492] T f53c6d2017c143e7bb396862fbe7a957 P a6ff3db459444683972a22263d29009b: Bootstrap starting.
I20260926 07:36:07.430905 13518 tablet_bootstrap.cc:492] T f53c6d2017c143e7bb396862fbe7a957 P 16209ececd2d420588b390f95db9b92a: Bootstrap starting.
I20260926 07:36:07.440766 13519 tablet_bootstrap.cc:654] T f53c6d2017c143e7bb396862fbe7a957 P a6ff3db459444683972a22263d29009b: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:07.442880 13519 log.cc:824] T f53c6d2017c143e7bb396862fbe7a957 P a6ff3db459444683972a22263d29009b: Log is configured to *not* fsync() on all Append() calls
I20260926 07:36:07.445925 13518 tablet_bootstrap.cc:654] T f53c6d2017c143e7bb396862fbe7a957 P 16209ececd2d420588b390f95db9b92a: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:07.448079 13518 log.cc:824] T f53c6d2017c143e7bb396862fbe7a957 P 16209ececd2d420588b390f95db9b92a: Log is configured to *not* fsync() on all Append() calls
I20260926 07:36:07.448773 13520 tablet_bootstrap.cc:492] T f53c6d2017c143e7bb396862fbe7a957 P 095560781b2d4e58b3804fce75312b90: Bootstrap starting.
I20260926 07:36:07.456012 13519 tablet_bootstrap.cc:492] T f53c6d2017c143e7bb396862fbe7a957 P a6ff3db459444683972a22263d29009b: No bootstrap required, opened a new log
I20260926 07:36:07.456583 13519 ts_tablet_manager.cc:1397] T f53c6d2017c143e7bb396862fbe7a957 P a6ff3db459444683972a22263d29009b: Time spent bootstrapping tablet: real 0.030s user 0.008s sys 0.006s
I20260926 07:36:07.466795 13518 tablet_bootstrap.cc:492] T f53c6d2017c143e7bb396862fbe7a957 P 16209ececd2d420588b390f95db9b92a: No bootstrap required, opened a new log
I20260926 07:36:07.467435 13518 ts_tablet_manager.cc:1397] T f53c6d2017c143e7bb396862fbe7a957 P 16209ececd2d420588b390f95db9b92a: Time spent bootstrapping tablet: real 0.039s user 0.010s sys 0.007s
I20260926 07:36:07.468075 13520 tablet_bootstrap.cc:654] T f53c6d2017c143e7bb396862fbe7a957 P 095560781b2d4e58b3804fce75312b90: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:07.470371 13520 log.cc:824] T f53c6d2017c143e7bb396862fbe7a957 P 095560781b2d4e58b3804fce75312b90: Log is configured to *not* fsync() on all Append() calls
07:36:07.473 [HMS - ERROR - pool-6-thread-2] (RetryingHMSHandler.java:201) AlreadyExistsException(message:Table hive.default.table_d7f4c6a111734e628f4183118d11aaef already exists)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1839)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1786)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:2035)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at com.sun.proxy.$Proxy25.create_table_with_environment_context(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14800)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14784)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
I20260926 07:36:07.476009 12742 master_service.cc:946] ReplaceTablet: received request to replace tablet f53c6d2017c143e7bb396862fbe7a957 from {username='slave'} at 127.0.0.1:45538
I20260926 07:36:07.476310 12741 master_service.cc:946] ReplaceTablet: received request to replace tablet f53c6d2017c143e7bb396862fbe7a957 from {username='slave'} at 127.0.0.1:45538
W20260926 07:36:07.488395 12743 catalog_manager.cc:2144] Already present: failed to create HMS catalog entry for table [id=b6f30c923e8546e3bd2b3c196036e0c2]: failed to create Hive MetaStore table: TException - service has thrown: AlreadyExistsException(message=Table hive.default.table_d7f4c6a111734e628f4183118d11aaef already exists)
I20260926 07:36:07.496369 12743 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45538:
name: "default.Table_8019d82e72b54af899515d80cb8aa33e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:36:07.497988 12742 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375: Sending DeleteTablet for 3 replicas of tablet f53c6d2017c143e7bb396862fbe7a957
I20260926 07:36:07.501955 13244 tablet_service.cc:1514] Processing DeleteTablet for tablet f53c6d2017c143e7bb396862fbe7a957 with delete_type TABLET_DATA_DELETED (replaced by tablet 96ba88660bae46be8ff396a93012630c) from {username='slave'} at 127.0.0.1:53506
I20260926 07:36:07.506064 13394 tablet_service.cc:1514] Processing DeleteTablet for tablet f53c6d2017c143e7bb396862fbe7a957 with delete_type TABLET_DATA_DELETED (replaced by tablet 96ba88660bae46be8ff396a93012630c) from {username='slave'} at 127.0.0.1:45090
I20260926 07:36:07.507263 13088 tablet_service.cc:1514] Processing DeleteTablet for tablet f53c6d2017c143e7bb396862fbe7a957 with delete_type TABLET_DATA_DELETED (replaced by tablet 96ba88660bae46be8ff396a93012630c) from {username='slave'} at 127.0.0.1:34588
I20260926 07:36:07.516561 12741 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45538:
name: "default.Table_daf8f6d0f2c6493d879b9a3bfbd1cfad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 07:36:07.519613 12732 catalog_manager.cc:4821] TS 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): delete failed for tablet f53c6d2017c143e7bb396862fbe7a957 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f53c6d2017c143e7bb396862fbe7a957 already in progress: creating tablet
I20260926 07:36:07.534195 13087 tablet_service.cc:1467] Processing CreateTablet for tablet 96ba88660bae46be8ff396a93012630c (DEFAULT_TABLE table=default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:07.535359 13087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96ba88660bae46be8ff396a93012630c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:07.535526 13244 tablet_service.cc:1467] Processing CreateTablet for tablet 96ba88660bae46be8ff396a93012630c (DEFAULT_TABLE table=default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:07.536531 13244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96ba88660bae46be8ff396a93012630c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:07.536654 13393 tablet_service.cc:1467] Processing CreateTablet for tablet 96ba88660bae46be8ff396a93012630c (DEFAULT_TABLE table=default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:07.537974 13393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96ba88660bae46be8ff396a93012630c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:07.548272 13519 raft_consensus.cc:348] T f53c6d2017c143e7bb396862fbe7a957 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:07.548964 13461 heartbeater.cc:503] Master 127.10.171.62:36693 was elected leader, sending a full tablet report...
I20260926 07:36:07.549206 13519 raft_consensus.cc:374] T f53c6d2017c143e7bb396862fbe7a957 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:07.549522 13519 raft_consensus.cc:729] T f53c6d2017c143e7bb396862fbe7a957 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ff3db459444683972a22263d29009b, State: Initialized, Role: FOLLOWER
I20260926 07:36:07.553545 12742 catalog_manager.cc:6565] ReplaceTablet: tablet f53c6d2017c143e7bb396862fbe7a957 deleted and replaced by tablet 96ba88660bae46be8ff396a93012630c
I20260926 07:36:07.559105 13519 consensus_queue.cc:260] T f53c6d2017c143e7bb396862fbe7a957 P a6ff3db459444683972a22263d29009b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:07.564093 13518 raft_consensus.cc:348] T f53c6d2017c143e7bb396862fbe7a957 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:07.565106 13518 raft_consensus.cc:374] T f53c6d2017c143e7bb396862fbe7a957 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:07.565407 13518 raft_consensus.cc:729] T f53c6d2017c143e7bb396862fbe7a957 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16209ececd2d420588b390f95db9b92a, State: Initialized, Role: FOLLOWER
I20260926 07:36:07.568964 13158 heartbeater.cc:503] Master 127.10.171.62:36693 was elected leader, sending a full tablet report...
I20260926 07:36:07.571019 10924 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
/tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.171.60:45615
--webserver_interface=127.10.171.60
--webserver_port=40625
--builtin_ntp_servers=127.10.171.20:42033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615
--hive_metastore_uris=thrift://127.0.0.1:35621
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 07:36:07.581624 13520 tablet_bootstrap.cc:492] T f53c6d2017c143e7bb396862fbe7a957 P 095560781b2d4e58b3804fce75312b90: No bootstrap required, opened a new log
I20260926 07:36:07.582285 13520 ts_tablet_manager.cc:1397] T f53c6d2017c143e7bb396862fbe7a957 P 095560781b2d4e58b3804fce75312b90: Time spent bootstrapping tablet: real 0.134s user 0.023s sys 0.007s
I20260926 07:36:07.582305 13518 consensus_queue.cc:260] T f53c6d2017c143e7bb396862fbe7a957 P 16209ececd2d420588b390f95db9b92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:07.591867 13519 ts_tablet_manager.cc:1428] T f53c6d2017c143e7bb396862fbe7a957 P a6ff3db459444683972a22263d29009b: Time spent starting tablet: real 0.135s user 0.032s sys 0.005s
W20260926 07:36:07.595955 12731 catalog_manager.cc:4821] TS a6ff3db459444683972a22263d29009b (127.10.171.3:34663): delete failed for tablet f53c6d2017c143e7bb396862fbe7a957 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f53c6d2017c143e7bb396862fbe7a957 already in progress: creating tablet
I20260926 07:36:07.604337 13394 tablet_service.cc:1514] Processing DeleteTablet for tablet f53c6d2017c143e7bb396862fbe7a957 with delete_type TABLET_DATA_DELETED (replaced by tablet 96ba88660bae46be8ff396a93012630c) from {username='slave'} at 127.0.0.1:45090
I20260926 07:36:07.608034 13311 heartbeater.cc:503] Master 127.10.171.62:36693 was elected leader, sending a full tablet report...
I20260926 07:36:07.611429 12744 catalog_manager.cc:2148] created HMS catalog entry for table [id=e093563ca611494caed51759972753c6]
I20260926 07:36:07.612083 13525 tablet_replica.cc:330] T f53c6d2017c143e7bb396862fbe7a957 P a6ff3db459444683972a22263d29009b: stopping tablet replica
I20260926 07:36:07.613018 13525 raft_consensus.cc:2229] T f53c6d2017c143e7bb396862fbe7a957 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:07.613598 13525 raft_consensus.cc:2258] T f53c6d2017c143e7bb396862fbe7a957 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Raft consensus is shut down!
W20260926 07:36:07.626564 12731 catalog_manager.cc:4821] TS 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): delete failed for tablet f53c6d2017c143e7bb396862fbe7a957 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f53c6d2017c143e7bb396862fbe7a957 already in progress: creating tablet
I20260926 07:36:07.639961 13244 tablet_service.cc:1514] Processing DeleteTablet for tablet f53c6d2017c143e7bb396862fbe7a957 with delete_type TABLET_DATA_DELETED (replaced by tablet 96ba88660bae46be8ff396a93012630c) from {username='slave'} at 127.0.0.1:53506
W20260926 07:36:07.642148 12732 catalog_manager.cc:4821] TS 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): delete failed for tablet f53c6d2017c143e7bb396862fbe7a957 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f53c6d2017c143e7bb396862fbe7a957 already in progress: creating tablet
I20260926 07:36:07.650321 13518 ts_tablet_manager.cc:1428] T f53c6d2017c143e7bb396862fbe7a957 P 16209ececd2d420588b390f95db9b92a: Time spent starting tablet: real 0.183s user 0.041s sys 0.023s
I20260926 07:36:07.651743 13519 tablet_bootstrap.cc:492] T 96ba88660bae46be8ff396a93012630c P a6ff3db459444683972a22263d29009b: Bootstrap starting.
I20260926 07:36:07.651644 12744 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45538:
table { table_name: "default.Table_5280e066b7404013a7c7bdfe8e9bc18b" } new_table_name: "default.Table_b2944ae24c6f47d4a1c3cb780994e4fe" modify_external_catalogs: true
I20260926 07:36:07.653179 13518 tablet_bootstrap.cc:492] T 96ba88660bae46be8ff396a93012630c P 16209ececd2d420588b390f95db9b92a: Bootstrap starting.
I20260926 07:36:07.656173 13244 tablet_service.cc:1514] Processing DeleteTablet for tablet f53c6d2017c143e7bb396862fbe7a957 with delete_type TABLET_DATA_DELETED (replaced by tablet 96ba88660bae46be8ff396a93012630c) from {username='slave'} at 127.0.0.1:53506
I20260926 07:36:07.658015 13525 ts_tablet_manager.cc:1905] T f53c6d2017c143e7bb396862fbe7a957 P a6ff3db459444683972a22263d29009b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:07.658761 13524 tablet_replica.cc:330] T f53c6d2017c143e7bb396862fbe7a957 P 16209ececd2d420588b390f95db9b92a: stopping tablet replica
I20260926 07:36:07.659577 13524 raft_consensus.cc:2229] T f53c6d2017c143e7bb396862fbe7a957 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:07.660245 13524 raft_consensus.cc:2258] T f53c6d2017c143e7bb396862fbe7a957 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:07.663196 13519 tablet_bootstrap.cc:654] T 96ba88660bae46be8ff396a93012630c P a6ff3db459444683972a22263d29009b: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:07.663326 13524 ts_tablet_manager.cc:1905] T f53c6d2017c143e7bb396862fbe7a957 P 16209ececd2d420588b390f95db9b92a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:07.665092 13518 tablet_bootstrap.cc:654] T 96ba88660bae46be8ff396a93012630c P 16209ececd2d420588b390f95db9b92a: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:07.671211 13518 tablet_bootstrap.cc:492] T 96ba88660bae46be8ff396a93012630c P 16209ececd2d420588b390f95db9b92a: No bootstrap required, opened a new log
I20260926 07:36:07.671640 13518 ts_tablet_manager.cc:1397] T 96ba88660bae46be8ff396a93012630c P 16209ececd2d420588b390f95db9b92a: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.007s
I20260926 07:36:07.674357 13518 raft_consensus.cc:348] T 96ba88660bae46be8ff396a93012630c P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:07.675120 13518 raft_consensus.cc:374] T 96ba88660bae46be8ff396a93012630c P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:07.675441 13518 raft_consensus.cc:729] T 96ba88660bae46be8ff396a93012630c P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16209ececd2d420588b390f95db9b92a, State: Initialized, Role: FOLLOWER
I20260926 07:36:07.676029 13518 consensus_queue.cc:260] T 96ba88660bae46be8ff396a93012630c P 16209ececd2d420588b390f95db9b92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:07.678053 13518 ts_tablet_manager.cc:1428] T 96ba88660bae46be8ff396a93012630c P 16209ececd2d420588b390f95db9b92a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 07:36:07.689857 13244 tablet_service.cc:1467] Processing CreateTablet for tablet ea121332fb4b464cb85defdff7abe758 (DEFAULT_TABLE table=default.table_5280e066b7404013a7c7bdfe8e9bc18b [id=e093563ca611494caed51759972753c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:07.690013 13525 ts_tablet_manager.cc:1918] T f53c6d2017c143e7bb396862fbe7a957 P a6ff3db459444683972a22263d29009b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:07.690426 13525 log.cc:1196] T f53c6d2017c143e7bb396862fbe7a957 P a6ff3db459444683972a22263d29009b: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/f53c6d2017c143e7bb396862fbe7a957
I20260926 07:36:07.690904 13244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea121332fb4b464cb85defdff7abe758. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:07.696076 13525 ts_tablet_manager.cc:1939] T f53c6d2017c143e7bb396862fbe7a957 P a6ff3db459444683972a22263d29009b: Deleting consensus metadata
I20260926 07:36:07.699144 12731 catalog_manager.cc:4841] TS a6ff3db459444683972a22263d29009b (127.10.171.3:34663): tablet f53c6d2017c143e7bb396862fbe7a957 (table default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]) successfully deleted
I20260926 07:36:07.700482 13088 tablet_service.cc:1467] Processing CreateTablet for tablet ea121332fb4b464cb85defdff7abe758 (DEFAULT_TABLE table=default.table_5280e066b7404013a7c7bdfe8e9bc18b [id=e093563ca611494caed51759972753c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:07.700641 13393 tablet_service.cc:1467] Processing CreateTablet for tablet ea121332fb4b464cb85defdff7abe758 (DEFAULT_TABLE table=default.table_5280e066b7404013a7c7bdfe8e9bc18b [id=e093563ca611494caed51759972753c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:07.701994 13393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea121332fb4b464cb85defdff7abe758. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:07.704562 13088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea121332fb4b464cb85defdff7abe758. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:07.710417 13524 ts_tablet_manager.cc:1918] T f53c6d2017c143e7bb396862fbe7a957 P 16209ececd2d420588b390f95db9b92a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:07.710835 13524 log.cc:1196] T f53c6d2017c143e7bb396862fbe7a957 P 16209ececd2d420588b390f95db9b92a: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/f53c6d2017c143e7bb396862fbe7a957
I20260926 07:36:07.711707 13524 ts_tablet_manager.cc:1939] T f53c6d2017c143e7bb396862fbe7a957 P 16209ececd2d420588b390f95db9b92a: Deleting consensus metadata
I20260926 07:36:07.720403 12742 master_service.cc:946] ReplaceTablet: received request to replace tablet 96ba88660bae46be8ff396a93012630c from {username='slave'} at 127.0.0.1:45538
I20260926 07:36:07.726747 13518 tablet_bootstrap.cc:492] T ea121332fb4b464cb85defdff7abe758 P 16209ececd2d420588b390f95db9b92a: Bootstrap starting.
I20260926 07:36:07.732390 13518 tablet_bootstrap.cc:654] T ea121332fb4b464cb85defdff7abe758 P 16209ececd2d420588b390f95db9b92a: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:07.737418 12732 catalog_manager.cc:4841] TS 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): tablet f53c6d2017c143e7bb396862fbe7a957 (table default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]) successfully deleted
I20260926 07:36:07.751520 13518 tablet_bootstrap.cc:492] T ea121332fb4b464cb85defdff7abe758 P 16209ececd2d420588b390f95db9b92a: No bootstrap required, opened a new log
I20260926 07:36:07.751900 12742 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375: Sending DeleteTablet for 3 replicas of tablet 96ba88660bae46be8ff396a93012630c
I20260926 07:36:07.751967 13518 ts_tablet_manager.cc:1397] T ea121332fb4b464cb85defdff7abe758 P 16209ececd2d420588b390f95db9b92a: Time spent bootstrapping tablet: real 0.025s user 0.013s sys 0.003s
I20260926 07:36:07.754953 13244 tablet_service.cc:1514] Processing DeleteTablet for tablet 96ba88660bae46be8ff396a93012630c with delete_type TABLET_DATA_DELETED (replaced by tablet 06fca4a37fa74d0893b0568db7f399b1) from {username='slave'} at 127.0.0.1:53506
I20260926 07:36:07.755679 13524 tablet_replica.cc:330] T 96ba88660bae46be8ff396a93012630c P 16209ececd2d420588b390f95db9b92a: stopping tablet replica
I20260926 07:36:07.756410 13524 raft_consensus.cc:2229] T 96ba88660bae46be8ff396a93012630c P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:07.756528 12742 catalog_manager.cc:6565] ReplaceTablet: tablet 96ba88660bae46be8ff396a93012630c deleted and replaced by tablet 06fca4a37fa74d0893b0568db7f399b1
I20260926 07:36:07.756858 13524 raft_consensus.cc:2258] T 96ba88660bae46be8ff396a93012630c P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:07.759547 13524 ts_tablet_manager.cc:1905] T 96ba88660bae46be8ff396a93012630c P 16209ececd2d420588b390f95db9b92a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:07.766259 13087 tablet_service.cc:1514] Processing DeleteTablet for tablet 96ba88660bae46be8ff396a93012630c with delete_type TABLET_DATA_DELETED (replaced by tablet 06fca4a37fa74d0893b0568db7f399b1) from {username='slave'} at 127.0.0.1:34588
I20260926 07:36:07.767148 13394 tablet_service.cc:1514] Processing DeleteTablet for tablet 96ba88660bae46be8ff396a93012630c with delete_type TABLET_DATA_DELETED (replaced by tablet 06fca4a37fa74d0893b0568db7f399b1) from {username='slave'} at 127.0.0.1:45090
W20260926 07:36:07.768244 12731 catalog_manager.cc:4821] TS 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): delete failed for tablet 96ba88660bae46be8ff396a93012630c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 96ba88660bae46be8ff396a93012630c already in progress: creating tablet
I20260926 07:36:07.776005 12742 master_service.cc:946] ReplaceTablet: received request to replace tablet f53c6d2017c143e7bb396862fbe7a957 from {username='slave'} at 127.0.0.1:45538
W20260926 07:36:07.776095 12731 catalog_manager.cc:4821] TS a6ff3db459444683972a22263d29009b (127.10.171.3:34663): delete failed for tablet 96ba88660bae46be8ff396a93012630c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 96ba88660bae46be8ff396a93012630c already in progress: creating tablet
I20260926 07:36:07.778733 13520 raft_consensus.cc:348] T f53c6d2017c143e7bb396862fbe7a957 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:07.779671 13520 raft_consensus.cc:374] T f53c6d2017c143e7bb396862fbe7a957 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:07.755645 13518 raft_consensus.cc:348] T ea121332fb4b464cb85defdff7abe758 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:07.780894 13518 raft_consensus.cc:374] T ea121332fb4b464cb85defdff7abe758 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:07.781207 13518 raft_consensus.cc:729] T ea121332fb4b464cb85defdff7abe758 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16209ececd2d420588b390f95db9b92a, State: Initialized, Role: FOLLOWER
I20260926 07:36:07.784090 13518 consensus_queue.cc:260] T ea121332fb4b464cb85defdff7abe758 P 16209ececd2d420588b390f95db9b92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:07.788522 13088 tablet_service.cc:1467] Processing CreateTablet for tablet 06fca4a37fa74d0893b0568db7f399b1 (DEFAULT_TABLE table=default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:07.789415 13393 tablet_service.cc:1467] Processing CreateTablet for tablet 06fca4a37fa74d0893b0568db7f399b1 (DEFAULT_TABLE table=default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:07.789762 13088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06fca4a37fa74d0893b0568db7f399b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:07.791002 13393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06fca4a37fa74d0893b0568db7f399b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:07.793320 13244 tablet_service.cc:1467] Processing CreateTablet for tablet 06fca4a37fa74d0893b0568db7f399b1 (DEFAULT_TABLE table=default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:07.790161 13518 ts_tablet_manager.cc:1428] T ea121332fb4b464cb85defdff7abe758 P 16209ececd2d420588b390f95db9b92a: Time spent starting tablet: real 0.038s user 0.002s sys 0.004s
I20260926 07:36:07.794358 13244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06fca4a37fa74d0893b0568db7f399b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:07.797869 13519 tablet_bootstrap.cc:492] T 96ba88660bae46be8ff396a93012630c P a6ff3db459444683972a22263d29009b: No bootstrap required, opened a new log
I20260926 07:36:07.798296 13519 ts_tablet_manager.cc:1397] T 96ba88660bae46be8ff396a93012630c P a6ff3db459444683972a22263d29009b: Time spent bootstrapping tablet: real 0.147s user 0.023s sys 0.015s
I20260926 07:36:07.801019 13519 raft_consensus.cc:348] T 96ba88660bae46be8ff396a93012630c P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:07.801748 13519 raft_consensus.cc:374] T 96ba88660bae46be8ff396a93012630c P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:07.812368 13519 raft_consensus.cc:729] T 96ba88660bae46be8ff396a93012630c P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ff3db459444683972a22263d29009b, State: Initialized, Role: FOLLOWER
I20260926 07:36:07.813235 13519 consensus_queue.cc:260] T 96ba88660bae46be8ff396a93012630c P a6ff3db459444683972a22263d29009b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:07.779989 13520 raft_consensus.cc:729] T f53c6d2017c143e7bb396862fbe7a957 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 095560781b2d4e58b3804fce75312b90, State: Initialized, Role: FOLLOWER
I20260926 07:36:07.815181 13520 consensus_queue.cc:260] T f53c6d2017c143e7bb396862fbe7a957 P 095560781b2d4e58b3804fce75312b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:07.821265 13518 tablet_bootstrap.cc:492] T 06fca4a37fa74d0893b0568db7f399b1 P 16209ececd2d420588b390f95db9b92a: Bootstrap starting.
W20260926 07:36:07.824515 13312 tablet.cc:2369] T ea121332fb4b464cb85defdff7abe758 P 16209ececd2d420588b390f95db9b92a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:36:07.827468 13519 ts_tablet_manager.cc:1428] T 96ba88660bae46be8ff396a93012630c P a6ff3db459444683972a22263d29009b: Time spent starting tablet: real 0.029s user 0.003s sys 0.004s
I20260926 07:36:07.834930 13519 tablet_bootstrap.cc:492] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b: Bootstrap starting.
I20260926 07:36:07.836567 13518 tablet_bootstrap.cc:654] T 06fca4a37fa74d0893b0568db7f399b1 P 16209ececd2d420588b390f95db9b92a: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:07.840198 13524 ts_tablet_manager.cc:1918] T 96ba88660bae46be8ff396a93012630c P 16209ececd2d420588b390f95db9b92a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:07.840618 13524 log.cc:1196] T 96ba88660bae46be8ff396a93012630c P 16209ececd2d420588b390f95db9b92a: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/96ba88660bae46be8ff396a93012630c
I20260926 07:36:07.841293 13524 ts_tablet_manager.cc:1939] T 96ba88660bae46be8ff396a93012630c P 16209ececd2d420588b390f95db9b92a: Deleting consensus metadata
I20260926 07:36:07.845495 13520 ts_tablet_manager.cc:1428] T f53c6d2017c143e7bb396862fbe7a957 P 095560781b2d4e58b3804fce75312b90: Time spent starting tablet: real 0.263s user 0.034s sys 0.003s
I20260926 07:36:07.846448 13520 tablet_bootstrap.cc:492] T 96ba88660bae46be8ff396a93012630c P 095560781b2d4e58b3804fce75312b90: Bootstrap starting.
I20260926 07:36:07.854043 13519 tablet_bootstrap.cc:654] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:07.855270 12746 catalog_manager.cc:2148] created HMS catalog entry for table [id=5eab828232e7481fa0f830a29a32f1b6]
I20260926 07:36:07.858247 13520 tablet_bootstrap.cc:654] T 96ba88660bae46be8ff396a93012630c P 095560781b2d4e58b3804fce75312b90: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:07.862912 12742 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45538:
name: "default.Table_6d097d04d5e6412d9f5354f5c40d580f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:36:07.868444 13088 tablet_service.cc:1514] Processing DeleteTablet for tablet f53c6d2017c143e7bb396862fbe7a957 with delete_type TABLET_DATA_DELETED (replaced by tablet 96ba88660bae46be8ff396a93012630c) from {username='slave'} at 127.0.0.1:34588
I20260926 07:36:07.868713 12732 catalog_manager.cc:4841] TS 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): tablet 96ba88660bae46be8ff396a93012630c (table default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]) successfully deleted
I20260926 07:36:07.869390 13526 tablet_replica.cc:330] T f53c6d2017c143e7bb396862fbe7a957 P 095560781b2d4e58b3804fce75312b90: stopping tablet replica
I20260926 07:36:07.870232 13526 raft_consensus.cc:2229] T f53c6d2017c143e7bb396862fbe7a957 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:07.870810 13526 raft_consensus.cc:2258] T f53c6d2017c143e7bb396862fbe7a957 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:07.873786 13526 ts_tablet_manager.cc:1905] T f53c6d2017c143e7bb396862fbe7a957 P 095560781b2d4e58b3804fce75312b90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:07.875533 13520 tablet_bootstrap.cc:492] T 96ba88660bae46be8ff396a93012630c P 095560781b2d4e58b3804fce75312b90: No bootstrap required, opened a new log
I20260926 07:36:07.875941 13520 ts_tablet_manager.cc:1397] T 96ba88660bae46be8ff396a93012630c P 095560781b2d4e58b3804fce75312b90: Time spent bootstrapping tablet: real 0.030s user 0.008s sys 0.005s
I20260926 07:36:07.882477 13393 tablet_service.cc:1514] Processing DeleteTablet for tablet 96ba88660bae46be8ff396a93012630c with delete_type TABLET_DATA_DELETED (replaced by tablet 06fca4a37fa74d0893b0568db7f399b1) from {username='slave'} at 127.0.0.1:45090
I20260926 07:36:07.883747 13519 tablet_bootstrap.cc:492] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b: No bootstrap required, opened a new log
I20260926 07:36:07.884177 13519 ts_tablet_manager.cc:1397] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b: Time spent bootstrapping tablet: real 0.049s user 0.014s sys 0.000s
I20260926 07:36:07.884577 13525 tablet_replica.cc:330] T 96ba88660bae46be8ff396a93012630c P a6ff3db459444683972a22263d29009b: stopping tablet replica
I20260926 07:36:07.885293 13525 raft_consensus.cc:2229] T 96ba88660bae46be8ff396a93012630c P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:07.885735 13525 raft_consensus.cc:2258] T 96ba88660bae46be8ff396a93012630c P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:07.886001 12739 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45538:
table { table_name: "default.Table_b9246fcaf5ed4103b8c7835af689efab" } modify_external_catalogs: true
I20260926 07:36:07.886765 12739 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45538:
table { table_name: "default.Table_b9246fcaf5ed4103b8c7835af689efab" } modify_external_catalogs: true
I20260926 07:36:07.886901 13519 raft_consensus.cc:348] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:07.887629 13519 raft_consensus.cc:374] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:07.890372 13518 tablet_bootstrap.cc:492] T 06fca4a37fa74d0893b0568db7f399b1 P 16209ececd2d420588b390f95db9b92a: No bootstrap required, opened a new log
I20260926 07:36:07.890829 13518 ts_tablet_manager.cc:1397] T 06fca4a37fa74d0893b0568db7f399b1 P 16209ececd2d420588b390f95db9b92a: Time spent bootstrapping tablet: real 0.072s user 0.007s sys 0.004s
I20260926 07:36:07.893446 13518 raft_consensus.cc:348] T 06fca4a37fa74d0893b0568db7f399b1 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:07.894176 13518 raft_consensus.cc:374] T 06fca4a37fa74d0893b0568db7f399b1 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:07.894471 13518 raft_consensus.cc:729] T 06fca4a37fa74d0893b0568db7f399b1 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16209ececd2d420588b390f95db9b92a, State: Initialized, Role: FOLLOWER
I20260926 07:36:07.887897 13519 raft_consensus.cc:729] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ff3db459444683972a22263d29009b, State: Initialized, Role: FOLLOWER
I20260926 07:36:07.895334 13525 ts_tablet_manager.cc:1905] T 96ba88660bae46be8ff396a93012630c P a6ff3db459444683972a22263d29009b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:07.897094 13520 raft_consensus.cc:348] T 96ba88660bae46be8ff396a93012630c P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:07.895664 13519 consensus_queue.cc:260] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:07.897817 13520 raft_consensus.cc:374] T 96ba88660bae46be8ff396a93012630c P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:07.909449 13519 ts_tablet_manager.cc:1428] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b: Time spent starting tablet: real 0.025s user 0.005s sys 0.001s
I20260926 07:36:07.908530 13520 raft_consensus.cc:729] T 96ba88660bae46be8ff396a93012630c P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 095560781b2d4e58b3804fce75312b90, State: Initialized, Role: FOLLOWER
I20260926 07:36:07.910426 13520 consensus_queue.cc:260] T 96ba88660bae46be8ff396a93012630c P 095560781b2d4e58b3804fce75312b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:07.912437 13520 ts_tablet_manager.cc:1428] T 96ba88660bae46be8ff396a93012630c P 095560781b2d4e58b3804fce75312b90: Time spent starting tablet: real 0.036s user 0.006s sys 0.000s
I20260926 07:36:07.913344 13520 tablet_bootstrap.cc:492] T ea121332fb4b464cb85defdff7abe758 P 095560781b2d4e58b3804fce75312b90: Bootstrap starting.
I20260926 07:36:07.914574 13393 tablet_service.cc:1467] Processing CreateTablet for tablet 9060d09b80e948a5b36b1004ae4f2321 (DEFAULT_TABLE table=default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:07.915738 13393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9060d09b80e948a5b36b1004ae4f2321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:07.917899 13244 tablet_service.cc:1467] Processing CreateTablet for tablet 9060d09b80e948a5b36b1004ae4f2321 (DEFAULT_TABLE table=default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:07.919019 13244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9060d09b80e948a5b36b1004ae4f2321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:07.920825 13519 tablet_bootstrap.cc:492] T 06fca4a37fa74d0893b0568db7f399b1 P a6ff3db459444683972a22263d29009b: Bootstrap starting.
I20260926 07:36:07.924583 13518 consensus_queue.cc:260] T 06fca4a37fa74d0893b0568db7f399b1 P 16209ececd2d420588b390f95db9b92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
F20260926 07:36:07.925730 12974 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_708d07ef404b42629d7c958b81980ca1
*** Check failure stack trace: ***
I20260926 07:36:07.929121 13518 ts_tablet_manager.cc:1428] T 06fca4a37fa74d0893b0568db7f399b1 P 16209ececd2d420588b390f95db9b92a: Time spent starting tablet: real 0.038s user 0.005s sys 0.001s
I20260926 07:36:07.929733 13088 tablet_service.cc:1514] Processing DeleteTablet for tablet 96ba88660bae46be8ff396a93012630c with delete_type TABLET_DATA_DELETED (replaced by tablet 06fca4a37fa74d0893b0568db7f399b1) from {username='slave'} at 127.0.0.1:34588
I20260926 07:36:07.934374 13519 tablet_bootstrap.cc:654] T 06fca4a37fa74d0893b0568db7f399b1 P a6ff3db459444683972a22263d29009b: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:07.934123 13087 tablet_service.cc:1467] Processing CreateTablet for tablet 9060d09b80e948a5b36b1004ae4f2321 (DEFAULT_TABLE table=default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:07.935288 13087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9060d09b80e948a5b36b1004ae4f2321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:07.942256 13520 tablet_bootstrap.cc:654] T ea121332fb4b464cb85defdff7abe758 P 095560781b2d4e58b3804fce75312b90: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:07.955075 13518 tablet_bootstrap.cc:492] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a: Bootstrap starting.
I20260926 07:36:07.957916 13525 ts_tablet_manager.cc:1918] T 96ba88660bae46be8ff396a93012630c P a6ff3db459444683972a22263d29009b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:07.959551 13525 log.cc:1196] T 96ba88660bae46be8ff396a93012630c P a6ff3db459444683972a22263d29009b: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/96ba88660bae46be8ff396a93012630c
I20260926 07:36:07.960358 13525 ts_tablet_manager.cc:1939] T 96ba88660bae46be8ff396a93012630c P a6ff3db459444683972a22263d29009b: Deleting consensus metadata
I20260926 07:36:07.966256 12731 catalog_manager.cc:4841] TS a6ff3db459444683972a22263d29009b (127.10.171.3:34663): tablet 96ba88660bae46be8ff396a93012630c (table default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]) successfully deleted
I20260926 07:36:07.968043 13518 tablet_bootstrap.cc:654] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:07.975657 12746 master_service.cc:946] ReplaceTablet: received request to replace tablet f53c6d2017c143e7bb396862fbe7a957 from {username='slave'} at 127.0.0.1:45538
I20260926 07:36:07.981227 13526 ts_tablet_manager.cc:1918] T f53c6d2017c143e7bb396862fbe7a957 P 095560781b2d4e58b3804fce75312b90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:07.985706 12746 master_service.cc:946] ReplaceTablet: received request to replace tablet ea121332fb4b464cb85defdff7abe758 from {username='slave'} at 127.0.0.1:45538
I20260926 07:36:07.986249 13526 log.cc:1196] T f53c6d2017c143e7bb396862fbe7a957 P 095560781b2d4e58b3804fce75312b90: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/f53c6d2017c143e7bb396862fbe7a957
I20260926 07:36:07.989084 13526 ts_tablet_manager.cc:1939] T f53c6d2017c143e7bb396862fbe7a957 P 095560781b2d4e58b3804fce75312b90: Deleting consensus metadata
I20260926 07:36:07.993214 13526 tablet_replica.cc:330] T 96ba88660bae46be8ff396a93012630c P 095560781b2d4e58b3804fce75312b90: stopping tablet replica
I20260926 07:36:07.993746 12731 catalog_manager.cc:4841] TS 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): tablet f53c6d2017c143e7bb396862fbe7a957 (table default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]) successfully deleted
I20260926 07:36:07.997318 13519 tablet_bootstrap.cc:492] T 06fca4a37fa74d0893b0568db7f399b1 P a6ff3db459444683972a22263d29009b: No bootstrap required, opened a new log
I20260926 07:36:07.997774 13519 ts_tablet_manager.cc:1397] T 06fca4a37fa74d0893b0568db7f399b1 P a6ff3db459444683972a22263d29009b: Time spent bootstrapping tablet: real 0.077s user 0.010s sys 0.030s
I20260926 07:36:07.998795 13518 tablet_bootstrap.cc:492] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a: No bootstrap required, opened a new log
I20260926 07:36:07.999202 13518 ts_tablet_manager.cc:1397] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a: Time spent bootstrapping tablet: real 0.044s user 0.008s sys 0.002s
I20260926 07:36:08.000819 13519 raft_consensus.cc:348] T 06fca4a37fa74d0893b0568db7f399b1 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.001557 13519 raft_consensus.cc:374] T 06fca4a37fa74d0893b0568db7f399b1 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:08.001839 13519 raft_consensus.cc:729] T 06fca4a37fa74d0893b0568db7f399b1 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ff3db459444683972a22263d29009b, State: Initialized, Role: FOLLOWER
I20260926 07:36:08.001672 13518 raft_consensus.cc:348] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } }
W20260926 07:36:08.002090 13468 tablet.cc:2369] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:36:08.002368 13518 raft_consensus.cc:374] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:08.002650 13518 raft_consensus.cc:729] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16209ececd2d420588b390f95db9b92a, State: Initialized, Role: FOLLOWER
I20260926 07:36:08.003396 13518 consensus_queue.cc:260] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } }
I20260926 07:36:08.002755 13519 consensus_queue.cc:260] T 06fca4a37fa74d0893b0568db7f399b1 P a6ff3db459444683972a22263d29009b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.004561 13520 tablet_bootstrap.cc:492] T ea121332fb4b464cb85defdff7abe758 P 095560781b2d4e58b3804fce75312b90: No bootstrap required, opened a new log
I20260926 07:36:08.005590 13520 ts_tablet_manager.cc:1397] T ea121332fb4b464cb85defdff7abe758 P 095560781b2d4e58b3804fce75312b90: Time spent bootstrapping tablet: real 0.092s user 0.010s sys 0.017s
I20260926 07:36:08.008041 13520 raft_consensus.cc:348] T ea121332fb4b464cb85defdff7abe758 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:08.008765 13520 raft_consensus.cc:374] T ea121332fb4b464cb85defdff7abe758 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:08.009051 13520 raft_consensus.cc:729] T ea121332fb4b464cb85defdff7abe758 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 095560781b2d4e58b3804fce75312b90, State: Initialized, Role: FOLLOWER
I20260926 07:36:08.009729 13520 consensus_queue.cc:260] T ea121332fb4b464cb85defdff7abe758 P 095560781b2d4e58b3804fce75312b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:08.010633 13526 raft_consensus.cc:2229] T 96ba88660bae46be8ff396a93012630c P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:08.015704 13526 raft_consensus.cc:2258] T 96ba88660bae46be8ff396a93012630c P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260926 07:36:08.018183 13163 tablet.cc:2369] T ea121332fb4b464cb85defdff7abe758 P 095560781b2d4e58b3804fce75312b90: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 07:36:08.014146 13520 ts_tablet_manager.cc:1428] T ea121332fb4b464cb85defdff7abe758 P 095560781b2d4e58b3804fce75312b90: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260926 07:36:08.020839 13520 tablet_bootstrap.cc:492] T 06fca4a37fa74d0893b0568db7f399b1 P 095560781b2d4e58b3804fce75312b90: Bootstrap starting.
I20260926 07:36:08.023520 13087 tablet_service.cc:1514] Processing DeleteTablet for tablet ea121332fb4b464cb85defdff7abe758 with delete_type TABLET_DATA_DELETED (replaced by tablet 758385269fe544bfa148f596a1649143) from {username='slave'} at 127.0.0.1:34588
I20260926 07:36:08.024407 12746 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375: Sending DeleteTablet for 3 replicas of tablet ea121332fb4b464cb85defdff7abe758
I20260926 07:36:08.028083 13526 ts_tablet_manager.cc:1905] T 96ba88660bae46be8ff396a93012630c P 095560781b2d4e58b3804fce75312b90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:08.029886 13087 tablet_service.cc:1514] Processing DeleteTablet for tablet ea121332fb4b464cb85defdff7abe758 with delete_type TABLET_DATA_DELETED (replaced by tablet 758385269fe544bfa148f596a1649143) from {username='slave'} at 127.0.0.1:34588
I20260926 07:36:08.030743 13519 ts_tablet_manager.cc:1428] T 06fca4a37fa74d0893b0568db7f399b1 P a6ff3db459444683972a22263d29009b: Time spent starting tablet: real 0.033s user 0.006s sys 0.000s
I20260926 07:36:08.031286 13518 ts_tablet_manager.cc:1428] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a: Time spent starting tablet: real 0.032s user 0.007s sys 0.000s
I20260926 07:36:08.033528 13393 tablet_service.cc:1514] Processing DeleteTablet for tablet ea121332fb4b464cb85defdff7abe758 with delete_type TABLET_DATA_DELETED (replaced by tablet 758385269fe544bfa148f596a1649143) from {username='slave'} at 127.0.0.1:45090
I20260926 07:36:08.040413 12746 catalog_manager.cc:6565] ReplaceTablet: tablet ea121332fb4b464cb85defdff7abe758 deleted and replaced by tablet 758385269fe544bfa148f596a1649143
I20260926 07:36:08.041339 13244 tablet_service.cc:1514] Processing DeleteTablet for tablet ea121332fb4b464cb85defdff7abe758 with delete_type TABLET_DATA_DELETED (replaced by tablet 758385269fe544bfa148f596a1649143) from {username='slave'} at 127.0.0.1:53506
I20260926 07:36:08.043293 13524 tablet_replica.cc:330] T ea121332fb4b464cb85defdff7abe758 P 16209ececd2d420588b390f95db9b92a: stopping tablet replica
I20260926 07:36:08.044040 13524 raft_consensus.cc:2229] T ea121332fb4b464cb85defdff7abe758 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:08.044559 13524 raft_consensus.cc:2258] T ea121332fb4b464cb85defdff7abe758 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:08.045846 13528 raft_consensus.cc:482] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:08.046367 13528 raft_consensus.cc:504] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
@ 0x7efc2c1c39a0 google::LogMessage::Flush() at ??:0
I20260926 07:36:08.049187 13528 leader_election.cc:288] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
I20260926 07:36:08.057078 13524 ts_tablet_manager.cc:1905] T ea121332fb4b464cb85defdff7abe758 P 16209ececd2d420588b390f95db9b92a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:08.059460 13525 tablet_replica.cc:330] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b: stopping tablet replica
I20260926 07:36:08.060272 13525 raft_consensus.cc:2229] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:08.060801 13525 raft_consensus.cc:2258] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:08.065953 13525 ts_tablet_manager.cc:1905] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:08.072324 13519 tablet_bootstrap.cc:492] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b: Bootstrap starting.
@ 0x7efc2c1c370f google::LogMessage::~LogMessage() at ??:0
I20260926 07:36:08.083297 12746 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45538:
name: "default.Table_5af6599ea22546fdb4c598d6a124f28d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:36:08.103896 13087 tablet_service.cc:1467] Processing CreateTablet for tablet 758385269fe544bfa148f596a1649143 (DEFAULT_TABLE table=default.table_5280e066b7404013a7c7bdfe8e9bc18b [id=e093563ca611494caed51759972753c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:08.105175 13087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 758385269fe544bfa148f596a1649143. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:08.107324 13520 tablet_bootstrap.cc:654] T 06fca4a37fa74d0893b0568db7f399b1 P 095560781b2d4e58b3804fce75312b90: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:08.119220 13244 tablet_service.cc:1467] Processing CreateTablet for tablet 758385269fe544bfa148f596a1649143 (DEFAULT_TABLE table=default.table_5280e066b7404013a7c7bdfe8e9bc18b [id=e093563ca611494caed51759972753c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:08.120429 13244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 758385269fe544bfa148f596a1649143. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:08.121889 13519 tablet_bootstrap.cc:654] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:08.131781 13393 tablet_service.cc:1467] Processing CreateTablet for tablet 758385269fe544bfa148f596a1649143 (DEFAULT_TABLE table=default.table_5280e066b7404013a7c7bdfe8e9bc18b [id=e093563ca611494caed51759972753c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:08.132668 13520 tablet_bootstrap.cc:492] T 06fca4a37fa74d0893b0568db7f399b1 P 095560781b2d4e58b3804fce75312b90: No bootstrap required, opened a new log
I20260926 07:36:08.132819 13524 ts_tablet_manager.cc:1918] T ea121332fb4b464cb85defdff7abe758 P 16209ececd2d420588b390f95db9b92a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:08.132968 13393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 758385269fe544bfa148f596a1649143. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:08.133200 13524 log.cc:1196] T ea121332fb4b464cb85defdff7abe758 P 16209ececd2d420588b390f95db9b92a: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/ea121332fb4b464cb85defdff7abe758
I20260926 07:36:08.133239 13520 ts_tablet_manager.cc:1397] T 06fca4a37fa74d0893b0568db7f399b1 P 095560781b2d4e58b3804fce75312b90: Time spent bootstrapping tablet: real 0.113s user 0.006s sys 0.006s
I20260926 07:36:08.133873 13524 ts_tablet_manager.cc:1939] T ea121332fb4b464cb85defdff7abe758 P 16209ececd2d420588b390f95db9b92a: Deleting consensus metadata
I20260926 07:36:08.136593 12732 catalog_manager.cc:4841] TS 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): tablet ea121332fb4b464cb85defdff7abe758 (table default.table_5280e066b7404013a7c7bdfe8e9bc18b [id=e093563ca611494caed51759972753c6]) successfully deleted
I20260926 07:36:08.139845 13518 tablet_bootstrap.cc:492] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a: Bootstrap starting.
I20260926 07:36:08.144492 13525 ts_tablet_manager.cc:1918] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:08.144874 13525 log.cc:1196] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/ea121332fb4b464cb85defdff7abe758
I20260926 07:36:08.145526 13525 ts_tablet_manager.cc:1939] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b: Deleting consensus metadata
I20260926 07:36:08.146281 13518 tablet_bootstrap.cc:654] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:08.146690 13526 ts_tablet_manager.cc:1918] T 96ba88660bae46be8ff396a93012630c P 095560781b2d4e58b3804fce75312b90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:08.147092 13526 log.cc:1196] T 96ba88660bae46be8ff396a93012630c P 095560781b2d4e58b3804fce75312b90: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/96ba88660bae46be8ff396a93012630c
I20260926 07:36:08.147768 13526 ts_tablet_manager.cc:1939] T 96ba88660bae46be8ff396a93012630c P 095560781b2d4e58b3804fce75312b90: Deleting consensus metadata
I20260926 07:36:08.148057 12731 catalog_manager.cc:4841] TS a6ff3db459444683972a22263d29009b (127.10.171.3:34663): tablet ea121332fb4b464cb85defdff7abe758 (table default.table_5280e066b7404013a7c7bdfe8e9bc18b [id=e093563ca611494caed51759972753c6]) successfully deleted
@ 0x7efc3725eaea kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260926 07:36:08.150054 13519 tablet_bootstrap.cc:492] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b: No bootstrap required, opened a new log
I20260926 07:36:08.150468 13519 ts_tablet_manager.cc:1397] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b: Time spent bootstrapping tablet: real 0.078s user 0.011s sys 0.000s
I20260926 07:36:08.151114 13520 raft_consensus.cc:348] T 06fca4a37fa74d0893b0568db7f399b1 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.152060 13520 raft_consensus.cc:374] T 06fca4a37fa74d0893b0568db7f399b1 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:08.152434 13520 raft_consensus.cc:729] T 06fca4a37fa74d0893b0568db7f399b1 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 095560781b2d4e58b3804fce75312b90, State: Initialized, Role: FOLLOWER
I20260926 07:36:08.153028 13519 raft_consensus.cc:348] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } }
I20260926 07:36:08.153720 13519 raft_consensus.cc:374] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:08.153999 13519 raft_consensus.cc:729] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ff3db459444683972a22263d29009b, State: Initialized, Role: FOLLOWER
I20260926 07:36:08.154873 13529 raft_consensus.cc:482] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:08.155405 13529 raft_consensus.cc:504] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } }
I20260926 07:36:08.160974 13526 tablet_replica.cc:330] T ea121332fb4b464cb85defdff7abe758 P 095560781b2d4e58b3804fce75312b90: stopping tablet replica
I20260926 07:36:08.161012 12731 catalog_manager.cc:4841] TS 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): tablet 96ba88660bae46be8ff396a93012630c (table default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]) successfully deleted
I20260926 07:36:08.161666 13526 raft_consensus.cc:2229] T ea121332fb4b464cb85defdff7abe758 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:08.162109 13526 raft_consensus.cc:2258] T ea121332fb4b464cb85defdff7abe758 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:08.162117 13529 leader_election.cc:288] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6ff3db459444683972a22263d29009b (127.10.171.3:34663), 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253)
I20260926 07:36:08.163172 13519 consensus_queue.cc:260] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } }
I20260926 07:36:08.169848 13108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea121332fb4b464cb85defdff7abe758" candidate_uuid: "a6ff3db459444683972a22263d29009b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "095560781b2d4e58b3804fce75312b90" is_pre_election: true
I20260926 07:36:08.164727 13520 consensus_queue.cc:260] T 06fca4a37fa74d0893b0568db7f399b1 P 095560781b2d4e58b3804fce75312b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.171301 13528 raft_consensus.cc:482] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:08.171720 13528 raft_consensus.cc:504] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } }
W20260926 07:36:08.172395 13347 leader_election.cc:341] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:36:08.172770 13520 ts_tablet_manager.cc:1428] T 06fca4a37fa74d0893b0568db7f399b1 P 095560781b2d4e58b3804fce75312b90: Time spent starting tablet: real 0.039s user 0.003s sys 0.004s
I20260926 07:36:08.173679 13528 leader_election.cc:288] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565), 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253)
I20260926 07:36:08.176292 13108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9060d09b80e948a5b36b1004ae4f2321" candidate_uuid: "a6ff3db459444683972a22263d29009b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "095560781b2d4e58b3804fce75312b90" is_pre_election: true
I20260926 07:36:08.177371 13519 ts_tablet_manager.cc:1428] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b: Time spent starting tablet: real 0.027s user 0.005s sys 0.001s
I20260926 07:36:08.178442 13263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9060d09b80e948a5b36b1004ae4f2321" candidate_uuid: "a6ff3db459444683972a22263d29009b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16209ececd2d420588b390f95db9b92a" is_pre_election: true
I20260926 07:36:08.179219 13263 raft_consensus.cc:2454] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ff3db459444683972a22263d29009b in term 0.
W20260926 07:36:08.181233 13347 leader_election.cc:341] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:36:08.183825 13520 tablet_bootstrap.cc:492] T 9060d09b80e948a5b36b1004ae4f2321 P 095560781b2d4e58b3804fce75312b90: Bootstrap starting.
I20260926 07:36:08.187863 12737 master_service.cc:946] ReplaceTablet: received request to replace tablet 9060d09b80e948a5b36b1004ae4f2321 from {username='slave'} at 127.0.0.1:45538
W20260926 07:36:08.188211 13348 leader_election.cc:341] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: ea121332fb4b464cb85defdff7abe758
I20260926 07:36:08.188637 13348 leader_election.cc:302] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
W20260926 07:36:08.189577 13348 raft_consensus.cc:2670] T ea121332fb4b464cb85defdff7abe758 P a6ff3db459444683972a22263d29009b: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260926 07:36:08.193130 13348 leader_election.cc:302] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16209ececd2d420588b390f95db9b92a, a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90
I20260926 07:36:08.193867 13528 raft_consensus.cc:2781] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:36:08.194229 13528 raft_consensus.cc:482] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:36:08.194548 13528 raft_consensus.cc:3037] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:36:08.198210 13526 ts_tablet_manager.cc:1905] T ea121332fb4b464cb85defdff7abe758 P 095560781b2d4e58b3804fce75312b90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:08.209394 12737 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375: Sending DeleteTablet for 3 replicas of tablet 9060d09b80e948a5b36b1004ae4f2321
I20260926 07:36:08.210845 12737 catalog_manager.cc:6565] ReplaceTablet: tablet 9060d09b80e948a5b36b1004ae4f2321 deleted and replaced by tablet 932e147b05fe4e1b95942182562feffe
I20260926 07:36:08.213330 13244 tablet_service.cc:1514] Processing DeleteTablet for tablet 9060d09b80e948a5b36b1004ae4f2321 with delete_type TABLET_DATA_DELETED (replaced by tablet 932e147b05fe4e1b95942182562feffe) from {username='slave'} at 127.0.0.1:53506
I20260926 07:36:08.214448 13524 tablet_replica.cc:330] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a: stopping tablet replica
I20260926 07:36:08.215142 13524 raft_consensus.cc:2229] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:08.215561 13524 raft_consensus.cc:2258] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:08.217555 13524 ts_tablet_manager.cc:1905] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:08.225512 13520 tablet_bootstrap.cc:654] T 9060d09b80e948a5b36b1004ae4f2321 P 095560781b2d4e58b3804fce75312b90: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:08.230441 13518 tablet_bootstrap.cc:492] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a: No bootstrap required, opened a new log
I20260926 07:36:08.231417 13414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9060d09b80e948a5b36b1004ae4f2321" candidate_uuid: "16209ececd2d420588b390f95db9b92a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ff3db459444683972a22263d29009b" is_pre_election: true
I20260926 07:36:08.233927 13518 ts_tablet_manager.cc:1397] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a: Time spent bootstrapping tablet: real 0.094s user 0.019s sys 0.001s
I20260926 07:36:08.234191 13394 tablet_service.cc:1514] Processing DeleteTablet for tablet 9060d09b80e948a5b36b1004ae4f2321 with delete_type TABLET_DATA_DELETED (replaced by tablet 932e147b05fe4e1b95942182562feffe) from {username='slave'} at 127.0.0.1:45090
I20260926 07:36:08.234949 13525 tablet_replica.cc:330] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b: stopping tablet replica
I20260926 07:36:08.235240 13087 tablet_service.cc:1514] Processing DeleteTablet for tablet 9060d09b80e948a5b36b1004ae4f2321 with delete_type TABLET_DATA_DELETED (replaced by tablet 932e147b05fe4e1b95942182562feffe) from {username='slave'} at 127.0.0.1:34588
I20260926 07:36:08.219954 13528 raft_consensus.cc:504] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } }
I20260926 07:36:08.240322 13263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9060d09b80e948a5b36b1004ae4f2321" candidate_uuid: "a6ff3db459444683972a22263d29009b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16209ececd2d420588b390f95db9b92a"
W20260926 07:36:08.241619 13348 leader_election.cc:341] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Illegal state: Tablet not RUNNING: STOPPED
I20260926 07:36:08.242245 13526 ts_tablet_manager.cc:1918] T ea121332fb4b464cb85defdff7abe758 P 095560781b2d4e58b3804fce75312b90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:08.242611 13526 log.cc:1196] T ea121332fb4b464cb85defdff7abe758 P 095560781b2d4e58b3804fce75312b90: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/ea121332fb4b464cb85defdff7abe758
I20260926 07:36:08.243266 13526 ts_tablet_manager.cc:1939] T ea121332fb4b464cb85defdff7abe758 P 095560781b2d4e58b3804fce75312b90: Deleting consensus metadata
I20260926 07:36:08.243191 13394 tablet_service.cc:1467] Processing CreateTablet for tablet 932e147b05fe4e1b95942182562feffe (DEFAULT_TABLE table=default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:08.244246 13394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 932e147b05fe4e1b95942182562feffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:08.244309 13525 raft_consensus.cc:2229] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:08.245360 13244 tablet_service.cc:1467] Processing CreateTablet for tablet 932e147b05fe4e1b95942182562feffe (DEFAULT_TABLE table=default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:08.246462 13244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 932e147b05fe4e1b95942182562feffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:08.246901 13525 raft_consensus.cc:2258] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:08.249181 13087 tablet_service.cc:1467] Processing CreateTablet for tablet 932e147b05fe4e1b95942182562feffe (DEFAULT_TABLE table=default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:08.250262 13087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 932e147b05fe4e1b95942182562feffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:08.251730 12740 master_service.cc:946] ReplaceTablet: received request to replace tablet 06fca4a37fa74d0893b0568db7f399b1 from {username='slave'} at 127.0.0.1:45538
W20260926 07:36:08.261270 13193 leader_election.cc:341] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:36:08.270612 13528 leader_election.cc:288] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 election: Requested vote from peers 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565), 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253)
I20260926 07:36:08.271238 13108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9060d09b80e948a5b36b1004ae4f2321" candidate_uuid: "a6ff3db459444683972a22263d29009b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "095560781b2d4e58b3804fce75312b90"
I20260926 07:36:08.272080 12740 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375: Sending DeleteTablet for 3 replicas of tablet 06fca4a37fa74d0893b0568db7f399b1
W20260926 07:36:08.272433 13347 leader_election.cc:341] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:36:08.272881 13347 leader_election.cc:302] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
W20260926 07:36:08.273521 13347 raft_consensus.cc:2670] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260926 07:36:08.274853 13519 tablet_bootstrap.cc:492] T 758385269fe544bfa148f596a1649143 P a6ff3db459444683972a22263d29009b: Bootstrap starting.
I20260926 07:36:08.274760 13243 tablet_service.cc:1514] Processing DeleteTablet for tablet 06fca4a37fa74d0893b0568db7f399b1 with delete_type TABLET_DATA_DELETED (replaced by tablet 8f0fde61e21444eaa091c9c3eb7ae8ec) from {username='slave'} at 127.0.0.1:53506
I20260926 07:36:08.275708 12740 catalog_manager.cc:6565] ReplaceTablet: tablet 06fca4a37fa74d0893b0568db7f399b1 deleted and replaced by tablet 8f0fde61e21444eaa091c9c3eb7ae8ec
I20260926 07:36:08.278229 13392 tablet_service.cc:1514] Processing DeleteTablet for tablet 06fca4a37fa74d0893b0568db7f399b1 with delete_type TABLET_DATA_DELETED (replaced by tablet 8f0fde61e21444eaa091c9c3eb7ae8ec) from {username='slave'} at 127.0.0.1:45090
I20260926 07:36:08.279507 13108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9060d09b80e948a5b36b1004ae4f2321" candidate_uuid: "16209ececd2d420588b390f95db9b92a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "095560781b2d4e58b3804fce75312b90" is_pre_election: true
I20260926 07:36:08.281589 13088 tablet_service.cc:1514] Processing DeleteTablet for tablet 06fca4a37fa74d0893b0568db7f399b1 with delete_type TABLET_DATA_DELETED (replaced by tablet 8f0fde61e21444eaa091c9c3eb7ae8ec) from {username='slave'} at 127.0.0.1:34588
I20260926 07:36:08.282223 13518 raft_consensus.cc:348] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.283349 13525 ts_tablet_manager.cc:1905] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7efc3725e33c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
W20260926 07:36:08.284505 13193 leader_election.cc:341] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:36:08.288383 13193 leader_election.cc:302] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
W20260926 07:36:08.289108 13193 raft_consensus.cc:2670] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260926 07:36:08.289940 13519 tablet_bootstrap.cc:654] T 758385269fe544bfa148f596a1649143 P a6ff3db459444683972a22263d29009b: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:08.289940 13518 raft_consensus.cc:374] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:08.290017 13520 tablet_bootstrap.cc:492] T 9060d09b80e948a5b36b1004ae4f2321 P 095560781b2d4e58b3804fce75312b90: No bootstrap required, opened a new log
I20260926 07:36:08.290228 13518 raft_consensus.cc:729] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16209ececd2d420588b390f95db9b92a, State: Initialized, Role: FOLLOWER
I20260926 07:36:08.290521 13520 ts_tablet_manager.cc:1397] T 9060d09b80e948a5b36b1004ae4f2321 P 095560781b2d4e58b3804fce75312b90: Time spent bootstrapping tablet: real 0.107s user 0.007s sys 0.005s
I20260926 07:36:08.290915 13518 consensus_queue.cc:260] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.298902 13520 raft_consensus.cc:348] T 9060d09b80e948a5b36b1004ae4f2321 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } }
I20260926 07:36:08.299611 13520 raft_consensus.cc:374] T 9060d09b80e948a5b36b1004ae4f2321 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:08.299873 13520 raft_consensus.cc:729] T 9060d09b80e948a5b36b1004ae4f2321 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 095560781b2d4e58b3804fce75312b90, State: Initialized, Role: FOLLOWER
I20260926 07:36:08.300515 13520 consensus_queue.cc:260] T 9060d09b80e948a5b36b1004ae4f2321 P 095560781b2d4e58b3804fce75312b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } }
I20260926 07:36:08.301206 13518 ts_tablet_manager.cc:1428] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a: Time spent starting tablet: real 0.066s user 0.005s sys 0.000s
I20260926 07:36:08.301879 12731 catalog_manager.cc:4841] TS 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): tablet ea121332fb4b464cb85defdff7abe758 (table default.table_5280e066b7404013a7c7bdfe8e9bc18b [id=e093563ca611494caed51759972753c6]) successfully deleted
W20260926 07:36:08.303308 12731 catalog_manager.cc:4808] TS 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): delete failed for tablet ea121332fb4b464cb85defdff7abe758 because the tablet was not found. No further retry: Not found: Tablet not found: ea121332fb4b464cb85defdff7abe758
I20260926 07:36:08.305296 13526 tablet_replica.cc:330] T 06fca4a37fa74d0893b0568db7f399b1 P 095560781b2d4e58b3804fce75312b90: stopping tablet replica
W20260926 07:36:08.305229 12731 catalog_manager.cc:4821] TS 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): delete failed for tablet 9060d09b80e948a5b36b1004ae4f2321 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9060d09b80e948a5b36b1004ae4f2321 already in progress: creating tablet
I20260926 07:36:08.305908 13526 raft_consensus.cc:2229] T 06fca4a37fa74d0893b0568db7f399b1 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:08.309144 13088 tablet_service.cc:1467] Processing CreateTablet for tablet 8f0fde61e21444eaa091c9c3eb7ae8ec (DEFAULT_TABLE table=default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:08.310196 13088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f0fde61e21444eaa091c9c3eb7ae8ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:08.314956 13394 tablet_service.cc:1467] Processing CreateTablet for tablet 8f0fde61e21444eaa091c9c3eb7ae8ec (DEFAULT_TABLE table=default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:08.316037 13394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f0fde61e21444eaa091c9c3eb7ae8ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:08.319756 13244 tablet_service.cc:1467] Processing CreateTablet for tablet 8f0fde61e21444eaa091c9c3eb7ae8ec (DEFAULT_TABLE table=default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:08.320722 13086 tablet_service.cc:1514] Processing DeleteTablet for tablet 9060d09b80e948a5b36b1004ae4f2321 with delete_type TABLET_DATA_DELETED (replaced by tablet 932e147b05fe4e1b95942182562feffe) from {username='slave'} at 127.0.0.1:34588
I20260926 07:36:08.321250 13244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f0fde61e21444eaa091c9c3eb7ae8ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:08.324105 13518 tablet_bootstrap.cc:492] T 932e147b05fe4e1b95942182562feffe P 16209ececd2d420588b390f95db9b92a: Bootstrap starting.
I20260926 07:36:08.324438 13520 ts_tablet_manager.cc:1428] T 9060d09b80e948a5b36b1004ae4f2321 P 095560781b2d4e58b3804fce75312b90: Time spent starting tablet: real 0.034s user 0.006s sys 0.000s
I20260926 07:36:08.328616 13520 tablet_bootstrap.cc:492] T 758385269fe544bfa148f596a1649143 P 095560781b2d4e58b3804fce75312b90: Bootstrap starting.
I20260926 07:36:08.331619 13526 raft_consensus.cc:2258] T 06fca4a37fa74d0893b0568db7f399b1 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:08.333509 13519 tablet_bootstrap.cc:492] T 758385269fe544bfa148f596a1649143 P a6ff3db459444683972a22263d29009b: No bootstrap required, opened a new log
I20260926 07:36:08.333950 13519 ts_tablet_manager.cc:1397] T 758385269fe544bfa148f596a1649143 P a6ff3db459444683972a22263d29009b: Time spent bootstrapping tablet: real 0.059s user 0.005s sys 0.004s
I20260926 07:36:08.335765 13518 tablet_bootstrap.cc:654] T 932e147b05fe4e1b95942182562feffe P 16209ececd2d420588b390f95db9b92a: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:08.340306 13524 ts_tablet_manager.cc:1918] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:08.340675 13524 log.cc:1196] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/9060d09b80e948a5b36b1004ae4f2321
I20260926 07:36:08.341303 13524 ts_tablet_manager.cc:1939] T 9060d09b80e948a5b36b1004ae4f2321 P 16209ececd2d420588b390f95db9b92a: Deleting consensus metadata
I20260926 07:36:08.341444 13526 ts_tablet_manager.cc:1905] T 06fca4a37fa74d0893b0568db7f399b1 P 095560781b2d4e58b3804fce75312b90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:08.343873 12732 catalog_manager.cc:4841] TS 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): tablet 9060d09b80e948a5b36b1004ae4f2321 (table default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]) successfully deleted
I20260926 07:36:08.347548 13519 raft_consensus.cc:348] T 758385269fe544bfa148f596a1649143 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.348335 13519 raft_consensus.cc:374] T 758385269fe544bfa148f596a1649143 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:08.348512 13524 tablet_replica.cc:330] T 06fca4a37fa74d0893b0568db7f399b1 P 16209ececd2d420588b390f95db9b92a: stopping tablet replica
I20260926 07:36:08.348567 13519 raft_consensus.cc:729] T 758385269fe544bfa148f596a1649143 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ff3db459444683972a22263d29009b, State: Initialized, Role: FOLLOWER
I20260926 07:36:08.349255 13524 raft_consensus.cc:2229] T 06fca4a37fa74d0893b0568db7f399b1 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:08.349701 13524 raft_consensus.cc:2258] T 06fca4a37fa74d0893b0568db7f399b1 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:08.353103 13520 tablet_bootstrap.cc:654] T 758385269fe544bfa148f596a1649143 P 095560781b2d4e58b3804fce75312b90: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:08.355187 13524 ts_tablet_manager.cc:1905] T 06fca4a37fa74d0893b0568db7f399b1 P 16209ececd2d420588b390f95db9b92a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:08.361447 13525 ts_tablet_manager.cc:1918] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:08.364358 13520 tablet_bootstrap.cc:492] T 758385269fe544bfa148f596a1649143 P 095560781b2d4e58b3804fce75312b90: No bootstrap required, opened a new log
I20260926 07:36:08.364818 13520 ts_tablet_manager.cc:1397] T 758385269fe544bfa148f596a1649143 P 095560781b2d4e58b3804fce75312b90: Time spent bootstrapping tablet: real 0.036s user 0.008s sys 0.005s
I20260926 07:36:08.365811 13518 tablet_bootstrap.cc:492] T 932e147b05fe4e1b95942182562feffe P 16209ececd2d420588b390f95db9b92a: No bootstrap required, opened a new log
I20260926 07:36:08.366287 13518 ts_tablet_manager.cc:1397] T 932e147b05fe4e1b95942182562feffe P 16209ececd2d420588b390f95db9b92a: Time spent bootstrapping tablet: real 0.042s user 0.009s sys 0.002s
I20260926 07:36:08.367296 13520 raft_consensus.cc:348] T 758385269fe544bfa148f596a1649143 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.367966 13520 raft_consensus.cc:374] T 758385269fe544bfa148f596a1649143 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:08.368379 13520 raft_consensus.cc:729] T 758385269fe544bfa148f596a1649143 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 095560781b2d4e58b3804fce75312b90, State: Initialized, Role: FOLLOWER
I20260926 07:36:08.368692 13518 raft_consensus.cc:348] T 932e147b05fe4e1b95942182562feffe P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.369223 13525 log.cc:1196] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/9060d09b80e948a5b36b1004ae4f2321
I20260926 07:36:08.369115 13520 consensus_queue.cc:260] T 758385269fe544bfa148f596a1649143 P 095560781b2d4e58b3804fce75312b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.369953 13525 ts_tablet_manager.cc:1939] T 9060d09b80e948a5b36b1004ae4f2321 P a6ff3db459444683972a22263d29009b: Deleting consensus metadata
@ 0x7efc37261fd2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260926 07:36:08.372210 13518 raft_consensus.cc:374] T 932e147b05fe4e1b95942182562feffe P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:08.372462 13518 raft_consensus.cc:729] T 932e147b05fe4e1b95942182562feffe P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16209ececd2d420588b390f95db9b92a, State: Initialized, Role: FOLLOWER
I20260926 07:36:08.362176 13519 consensus_queue.cc:260] T 758385269fe544bfa148f596a1649143 P a6ff3db459444683972a22263d29009b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.373848 13518 consensus_queue.cc:260] T 932e147b05fe4e1b95942182562feffe P 16209ececd2d420588b390f95db9b92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.379544 13520 ts_tablet_manager.cc:1428] T 758385269fe544bfa148f596a1649143 P 095560781b2d4e58b3804fce75312b90: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260926 07:36:08.380508 13520 tablet_bootstrap.cc:492] T 932e147b05fe4e1b95942182562feffe P 095560781b2d4e58b3804fce75312b90: Bootstrap starting.
I20260926 07:36:08.387630 13519 ts_tablet_manager.cc:1428] T 758385269fe544bfa148f596a1649143 P a6ff3db459444683972a22263d29009b: Time spent starting tablet: real 0.053s user 0.005s sys 0.003s
I20260926 07:36:08.392984 13518 ts_tablet_manager.cc:1428] T 932e147b05fe4e1b95942182562feffe P 16209ececd2d420588b390f95db9b92a: Time spent starting tablet: real 0.026s user 0.000s sys 0.004s
I20260926 07:36:08.398096 13519 tablet_bootstrap.cc:492] T 932e147b05fe4e1b95942182562feffe P a6ff3db459444683972a22263d29009b: Bootstrap starting.
I20260926 07:36:08.402320 12731 catalog_manager.cc:4841] TS a6ff3db459444683972a22263d29009b (127.10.171.3:34663): tablet 9060d09b80e948a5b36b1004ae4f2321 (table default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]) successfully deleted
I20260926 07:36:08.403684 13520 tablet_bootstrap.cc:654] T 932e147b05fe4e1b95942182562feffe P 095560781b2d4e58b3804fce75312b90: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:08.410384 13518 tablet_bootstrap.cc:492] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 16209ececd2d420588b390f95db9b92a: Bootstrap starting.
I20260926 07:36:08.415428 13520 tablet_bootstrap.cc:492] T 932e147b05fe4e1b95942182562feffe P 095560781b2d4e58b3804fce75312b90: No bootstrap required, opened a new log
I20260926 07:36:08.415828 13520 ts_tablet_manager.cc:1397] T 932e147b05fe4e1b95942182562feffe P 095560781b2d4e58b3804fce75312b90: Time spent bootstrapping tablet: real 0.035s user 0.002s sys 0.013s
I20260926 07:36:08.418231 13520 raft_consensus.cc:348] T 932e147b05fe4e1b95942182562feffe P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.418843 13520 raft_consensus.cc:374] T 932e147b05fe4e1b95942182562feffe P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:08.419072 13520 raft_consensus.cc:729] T 932e147b05fe4e1b95942182562feffe P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 095560781b2d4e58b3804fce75312b90, State: Initialized, Role: FOLLOWER
I20260926 07:36:08.420389 13520 consensus_queue.cc:260] T 932e147b05fe4e1b95942182562feffe P 095560781b2d4e58b3804fce75312b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.421698 13525 tablet_replica.cc:330] T 06fca4a37fa74d0893b0568db7f399b1 P a6ff3db459444683972a22263d29009b: stopping tablet replica
I20260926 07:36:08.422382 13525 raft_consensus.cc:2229] T 06fca4a37fa74d0893b0568db7f399b1 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:08.422848 13525 raft_consensus.cc:2258] T 06fca4a37fa74d0893b0568db7f399b1 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:08.423226 13518 tablet_bootstrap.cc:654] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 16209ececd2d420588b390f95db9b92a: Neither blocks nor log segments found. Creating new log.
@ 0x7efc37261f8a _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260926 07:36:08.432165 13524 ts_tablet_manager.cc:1918] T 06fca4a37fa74d0893b0568db7f399b1 P 16209ececd2d420588b390f95db9b92a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:08.432524 13524 log.cc:1196] T 06fca4a37fa74d0893b0568db7f399b1 P 16209ececd2d420588b390f95db9b92a: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/06fca4a37fa74d0893b0568db7f399b1
I20260926 07:36:08.433118 13520 ts_tablet_manager.cc:1428] T 932e147b05fe4e1b95942182562feffe P 095560781b2d4e58b3804fce75312b90: Time spent starting tablet: real 0.017s user 0.006s sys 0.001s
I20260926 07:36:08.434028 13520 tablet_bootstrap.cc:492] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 095560781b2d4e58b3804fce75312b90: Bootstrap starting.
I20260926 07:36:08.433132 13524 ts_tablet_manager.cc:1939] T 06fca4a37fa74d0893b0568db7f399b1 P 16209ececd2d420588b390f95db9b92a: Deleting consensus metadata
I20260926 07:36:08.448735 13520 tablet_bootstrap.cc:654] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 095560781b2d4e58b3804fce75312b90: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:08.451207 13525 ts_tablet_manager.cc:1905] T 06fca4a37fa74d0893b0568db7f399b1 P a6ff3db459444683972a22263d29009b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:08.462345 13518 tablet_bootstrap.cc:492] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 16209ececd2d420588b390f95db9b92a: No bootstrap required, opened a new log
I20260926 07:36:08.462790 13518 ts_tablet_manager.cc:1397] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 16209ececd2d420588b390f95db9b92a: Time spent bootstrapping tablet: real 0.053s user 0.007s sys 0.005s
I20260926 07:36:08.426914 13519 tablet_bootstrap.cc:654] T 932e147b05fe4e1b95942182562feffe P a6ff3db459444683972a22263d29009b: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:08.466178 12732 catalog_manager.cc:4841] TS 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): tablet 06fca4a37fa74d0893b0568db7f399b1 (table default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]) successfully deleted
I20260926 07:36:08.467270 12737 master_service.cc:946] ReplaceTablet: received request to replace tablet 932e147b05fe4e1b95942182562feffe from {username='slave'} at 127.0.0.1:45538
I20260926 07:36:08.470937 13526 ts_tablet_manager.cc:1918] T 06fca4a37fa74d0893b0568db7f399b1 P 095560781b2d4e58b3804fce75312b90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:08.471299 13526 log.cc:1196] T 06fca4a37fa74d0893b0568db7f399b1 P 095560781b2d4e58b3804fce75312b90: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/06fca4a37fa74d0893b0568db7f399b1
I20260926 07:36:08.471310 13518 raft_consensus.cc:348] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.471930 13526 ts_tablet_manager.cc:1939] T 06fca4a37fa74d0893b0568db7f399b1 P 095560781b2d4e58b3804fce75312b90: Deleting consensus metadata
I20260926 07:36:08.471997 13518 raft_consensus.cc:374] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:08.472379 13518 raft_consensus.cc:729] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16209ececd2d420588b390f95db9b92a, State: Initialized, Role: FOLLOWER
I20260926 07:36:08.473199 13518 consensus_queue.cc:260] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 16209ececd2d420588b390f95db9b92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.486305 13518 ts_tablet_manager.cc:1428] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 16209ececd2d420588b390f95db9b92a: Time spent starting tablet: real 0.023s user 0.004s sys 0.003s
I20260926 07:36:08.490947 13529 raft_consensus.cc:482] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:08.491420 13529 raft_consensus.cc:504] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.493324 12737 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375: Sending DeleteTablet for 3 replicas of tablet 932e147b05fe4e1b95942182562feffe
I20260926 07:36:08.493824 13529 leader_election.cc:288] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
I20260926 07:36:08.494633 12737 catalog_manager.cc:6565] ReplaceTablet: tablet 932e147b05fe4e1b95942182562feffe deleted and replaced by tablet 37337c5502804878a77653b6d1964ea4
I20260926 07:36:08.496555 13244 tablet_service.cc:1514] Processing DeleteTablet for tablet 932e147b05fe4e1b95942182562feffe with delete_type TABLET_DATA_DELETED (replaced by tablet 37337c5502804878a77653b6d1964ea4) from {username='slave'} at 127.0.0.1:53506
I20260926 07:36:08.497802 13524 tablet_replica.cc:330] T 932e147b05fe4e1b95942182562feffe P 16209ececd2d420588b390f95db9b92a: stopping tablet replica
I20260926 07:36:08.498557 13524 raft_consensus.cc:2229] T 932e147b05fe4e1b95942182562feffe P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:08.499028 13524 raft_consensus.cc:2258] T 932e147b05fe4e1b95942182562feffe P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:08.501443 13108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758385269fe544bfa148f596a1649143" candidate_uuid: "16209ececd2d420588b390f95db9b92a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "095560781b2d4e58b3804fce75312b90" is_pre_election: true
I20260926 07:36:08.502175 13524 ts_tablet_manager.cc:1905] T 932e147b05fe4e1b95942182562feffe P 16209ececd2d420588b390f95db9b92a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:08.502169 13108 raft_consensus.cc:2454] T 758385269fe544bfa148f596a1649143 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16209ececd2d420588b390f95db9b92a in term 0.
I20260926 07:36:08.503140 13414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758385269fe544bfa148f596a1649143" candidate_uuid: "16209ececd2d420588b390f95db9b92a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ff3db459444683972a22263d29009b" is_pre_election: true
I20260926 07:36:08.503737 13414 raft_consensus.cc:2454] T 758385269fe544bfa148f596a1649143 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16209ececd2d420588b390f95db9b92a in term 0.
I20260926 07:36:08.507048 13087 tablet_service.cc:1514] Processing DeleteTablet for tablet 932e147b05fe4e1b95942182562feffe with delete_type TABLET_DATA_DELETED (replaced by tablet 37337c5502804878a77653b6d1964ea4) from {username='slave'} at 127.0.0.1:34588
I20260926 07:36:08.507706 13394 tablet_service.cc:1514] Processing DeleteTablet for tablet 932e147b05fe4e1b95942182562feffe with delete_type TABLET_DATA_DELETED (replaced by tablet 37337c5502804878a77653b6d1964ea4) from {username='slave'} at 127.0.0.1:45090
I20260926 07:36:08.508589 13193 leader_election.cc:302] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16209ececd2d420588b390f95db9b92a, a6ff3db459444683972a22263d29009b; no voters:
I20260926 07:36:08.510237 13519 tablet_bootstrap.cc:492] T 932e147b05fe4e1b95942182562feffe P a6ff3db459444683972a22263d29009b: No bootstrap required, opened a new log
@ 0x7efc37261f1a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
I20260926 07:36:08.510354 12737 master_service.cc:946] ReplaceTablet: received request to replace tablet 9060d09b80e948a5b36b1004ae4f2321 from {username='slave'} at 127.0.0.1:45538
I20260926 07:36:08.510747 13519 ts_tablet_manager.cc:1397] T 932e147b05fe4e1b95942182562feffe P a6ff3db459444683972a22263d29009b: Time spent bootstrapping tablet: real 0.113s user 0.008s sys 0.001s
I20260926 07:36:08.512549 13529 raft_consensus.cc:2781] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:36:08.512892 13529 raft_consensus.cc:482] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:36:08.513221 13529 raft_consensus.cc:3037] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:36:08.517583 13520 tablet_bootstrap.cc:492] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 095560781b2d4e58b3804fce75312b90: No bootstrap required, opened a new log
I20260926 07:36:08.517997 13520 ts_tablet_manager.cc:1397] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 095560781b2d4e58b3804fce75312b90: Time spent bootstrapping tablet: real 0.084s user 0.010s sys 0.008s
I20260926 07:36:08.520220 13520 raft_consensus.cc:348] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.520852 13520 raft_consensus.cc:374] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:08.521085 13520 raft_consensus.cc:729] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 095560781b2d4e58b3804fce75312b90, State: Initialized, Role: FOLLOWER
I20260926 07:36:08.521281 13524 ts_tablet_manager.cc:1918] T 932e147b05fe4e1b95942182562feffe P 16209ececd2d420588b390f95db9b92a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:08.521624 13524 log.cc:1196] T 932e147b05fe4e1b95942182562feffe P 16209ececd2d420588b390f95db9b92a: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/932e147b05fe4e1b95942182562feffe
I20260926 07:36:08.521765 13520 consensus_queue.cc:260] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 095560781b2d4e58b3804fce75312b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.522267 13524 ts_tablet_manager.cc:1939] T 932e147b05fe4e1b95942182562feffe P 16209ececd2d420588b390f95db9b92a: Deleting consensus metadata
I20260926 07:36:08.524653 13520 ts_tablet_manager.cc:1428] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 095560781b2d4e58b3804fce75312b90: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260926 07:36:08.526321 13519 raft_consensus.cc:348] T 932e147b05fe4e1b95942182562feffe P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.527014 13519 raft_consensus.cc:374] T 932e147b05fe4e1b95942182562feffe P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:08.527256 13519 raft_consensus.cc:729] T 932e147b05fe4e1b95942182562feffe P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ff3db459444683972a22263d29009b, State: Initialized, Role: FOLLOWER
I20260926 07:36:08.530345 12731 catalog_manager.cc:4841] TS 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): tablet 06fca4a37fa74d0893b0568db7f399b1 (table default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]) successfully deleted
I20260926 07:36:08.531432 13526 tablet_replica.cc:330] T 9060d09b80e948a5b36b1004ae4f2321 P 095560781b2d4e58b3804fce75312b90: stopping tablet replica
I20260926 07:36:08.536585 13526 raft_consensus.cc:2229] T 9060d09b80e948a5b36b1004ae4f2321 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:08.537055 13526 raft_consensus.cc:2258] T 9060d09b80e948a5b36b1004ae4f2321 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:08.545862 13244 tablet_service.cc:1467] Processing CreateTablet for tablet 37337c5502804878a77653b6d1964ea4 (DEFAULT_TABLE table=default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:08.545909 13087 tablet_service.cc:1467] Processing CreateTablet for tablet 37337c5502804878a77653b6d1964ea4 (DEFAULT_TABLE table=default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:08.546990 13087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37337c5502804878a77653b6d1964ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:08.547015 13244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37337c5502804878a77653b6d1964ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:08.527855 13519 consensus_queue.cc:260] T 932e147b05fe4e1b95942182562feffe P a6ff3db459444683972a22263d29009b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.556933 12732 catalog_manager.cc:4841] TS 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): tablet 932e147b05fe4e1b95942182562feffe (table default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]) successfully deleted
I20260926 07:36:08.560593 13519 ts_tablet_manager.cc:1428] T 932e147b05fe4e1b95942182562feffe P a6ff3db459444683972a22263d29009b: Time spent starting tablet: real 0.050s user 0.008s sys 0.000s
I20260926 07:36:08.562918 13520 tablet_bootstrap.cc:492] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90: Bootstrap starting.
I20260926 07:36:08.566084 13529 raft_consensus.cc:504] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.566747 13394 tablet_service.cc:1467] Processing CreateTablet for tablet 37337c5502804878a77653b6d1964ea4 (DEFAULT_TABLE table=default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:08.568994 13108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758385269fe544bfa148f596a1649143" candidate_uuid: "16209ececd2d420588b390f95db9b92a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "095560781b2d4e58b3804fce75312b90"
I20260926 07:36:08.569391 13520 tablet_bootstrap.cc:654] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:08.569587 13108 raft_consensus.cc:3037] T 758385269fe544bfa148f596a1649143 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Advancing to term 1
@ 0x7efc37261ee2 std::__1::__function::__alloc_func<>::operator()() at ??:0
I20260926 07:36:08.576905 13108 raft_consensus.cc:2454] T 758385269fe544bfa148f596a1649143 P 095560781b2d4e58b3804fce75312b90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16209ececd2d420588b390f95db9b92a in term 1.
I20260926 07:36:08.578128 13520 tablet_bootstrap.cc:492] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90: No bootstrap required, opened a new log
I20260926 07:36:08.578032 13193 leader_election.cc:302] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a; no voters:
I20260926 07:36:08.578527 13520 ts_tablet_manager.cc:1397] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.000s
I20260926 07:36:08.580806 13520 raft_consensus.cc:348] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.581482 13520 raft_consensus.cc:374] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:08.581741 13520 raft_consensus.cc:729] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 095560781b2d4e58b3804fce75312b90, State: Initialized, Role: FOLLOWER
I20260926 07:36:08.582247 13525 ts_tablet_manager.cc:1918] T 06fca4a37fa74d0893b0568db7f399b1 P a6ff3db459444683972a22263d29009b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:08.582685 13525 log.cc:1196] T 06fca4a37fa74d0893b0568db7f399b1 P a6ff3db459444683972a22263d29009b: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/06fca4a37fa74d0893b0568db7f399b1
I20260926 07:36:08.583364 13525 ts_tablet_manager.cc:1939] T 06fca4a37fa74d0893b0568db7f399b1 P a6ff3db459444683972a22263d29009b: Deleting consensus metadata
I20260926 07:36:08.582306 13520 consensus_queue.cc:260] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.585582 13520 ts_tablet_manager.cc:1428] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
I20260926 07:36:08.587383 13394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37337c5502804878a77653b6d1964ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:08.605808 13519 tablet_bootstrap.cc:492] T 8f0fde61e21444eaa091c9c3eb7ae8ec P a6ff3db459444683972a22263d29009b: Bootstrap starting.
I20260926 07:36:08.614876 13525 tablet_replica.cc:330] T 932e147b05fe4e1b95942182562feffe P a6ff3db459444683972a22263d29009b: stopping tablet replica
I20260926 07:36:08.615109 12731 catalog_manager.cc:4841] TS a6ff3db459444683972a22263d29009b (127.10.171.3:34663): tablet 06fca4a37fa74d0893b0568db7f399b1 (table default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]) successfully deleted
I20260926 07:36:08.615559 13525 raft_consensus.cc:2229] T 932e147b05fe4e1b95942182562feffe P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:08.617930 13519 tablet_bootstrap.cc:654] T 8f0fde61e21444eaa091c9c3eb7ae8ec P a6ff3db459444683972a22263d29009b: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:08.626112 13526 ts_tablet_manager.cc:1905] T 9060d09b80e948a5b36b1004ae4f2321 P 095560781b2d4e58b3804fce75312b90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:08.616024 13525 raft_consensus.cc:2258] T 932e147b05fe4e1b95942182562feffe P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:08.634246 13525 ts_tablet_manager.cc:1905] T 932e147b05fe4e1b95942182562feffe P a6ff3db459444683972a22263d29009b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:08.636940 13529 leader_election.cc:288] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 election: Requested vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
I20260926 07:36:08.637631 13529 raft_consensus.cc:2781] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:36:08.638175 13529 raft_consensus.cc:686] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [term 1 LEADER]: Becoming Leader. State: Replica: 16209ececd2d420588b390f95db9b92a, State: Running, Role: LEADER
W20260926 07:36:08.640672 13193 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.1:37253, user_credentials={real_user=slave}} blocked reactor thread for 62741.4us
I20260926 07:36:08.642771 13414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758385269fe544bfa148f596a1649143" candidate_uuid: "16209ececd2d420588b390f95db9b92a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ff3db459444683972a22263d29009b"
I20260926 07:36:08.643330 13414 raft_consensus.cc:3037] T 758385269fe544bfa148f596a1649143 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:36:08.645130 13518 tablet_bootstrap.cc:492] T 37337c5502804878a77653b6d1964ea4 P 16209ececd2d420588b390f95db9b92a: Bootstrap starting.
I20260926 07:36:08.639283 13529 consensus_queue.cc:237] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.650069 13518 tablet_bootstrap.cc:654] T 37337c5502804878a77653b6d1964ea4 P 16209ececd2d420588b390f95db9b92a: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:08.650154 13414 raft_consensus.cc:2454] T 758385269fe544bfa148f596a1649143 P a6ff3db459444683972a22263d29009b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16209ececd2d420588b390f95db9b92a in term 1.
I20260926 07:36:08.657859 13519 tablet_bootstrap.cc:492] T 8f0fde61e21444eaa091c9c3eb7ae8ec P a6ff3db459444683972a22263d29009b: No bootstrap required, opened a new log
I20260926 07:36:08.658265 13519 ts_tablet_manager.cc:1397] T 8f0fde61e21444eaa091c9c3eb7ae8ec P a6ff3db459444683972a22263d29009b: Time spent bootstrapping tablet: real 0.053s user 0.006s sys 0.003s
@ 0x7efc372611de std::__1::__function::__func<>::operator()() at ??:0
I20260926 07:36:08.660967 13519 raft_consensus.cc:348] T 8f0fde61e21444eaa091c9c3eb7ae8ec P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.661633 13519 raft_consensus.cc:374] T 8f0fde61e21444eaa091c9c3eb7ae8ec P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:08.661872 13519 raft_consensus.cc:729] T 8f0fde61e21444eaa091c9c3eb7ae8ec P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ff3db459444683972a22263d29009b, State: Initialized, Role: FOLLOWER
@ 0x580785 std::__1::__function::__value_func<>::operator()() at ??:?
I20260926 07:36:08.662488 13519 consensus_queue.cc:260] T 8f0fde61e21444eaa091c9c3eb7ae8ec P a6ff3db459444683972a22263d29009b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
@ 0x5805ba std::__1::function<>::operator()() at ??:?
I20260926 07:36:08.665639 13518 tablet_bootstrap.cc:492] T 37337c5502804878a77653b6d1964ea4 P 16209ececd2d420588b390f95db9b92a: No bootstrap required, opened a new log
I20260926 07:36:08.666046 13518 ts_tablet_manager.cc:1397] T 37337c5502804878a77653b6d1964ea4 P 16209ececd2d420588b390f95db9b92a: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.002s
I20260926 07:36:08.669847 13518 raft_consensus.cc:348] T 37337c5502804878a77653b6d1964ea4 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.670454 13518 raft_consensus.cc:374] T 37337c5502804878a77653b6d1964ea4 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:08.670674 13518 raft_consensus.cc:729] T 37337c5502804878a77653b6d1964ea4 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16209ececd2d420588b390f95db9b92a, State: Initialized, Role: FOLLOWER
I20260926 07:36:08.671197 13518 consensus_queue.cc:260] T 37337c5502804878a77653b6d1964ea4 P 16209ececd2d420588b390f95db9b92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.672921 13518 ts_tablet_manager.cc:1428] T 37337c5502804878a77653b6d1964ea4 P 16209ececd2d420588b390f95db9b92a: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260926 07:36:08.677456 13537 raft_consensus.cc:482] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:08.677911 13537 raft_consensus.cc:504] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.682452 13519 ts_tablet_manager.cc:1428] T 8f0fde61e21444eaa091c9c3eb7ae8ec P a6ff3db459444683972a22263d29009b: Time spent starting tablet: real 0.024s user 0.006s sys 0.001s
I20260926 07:36:08.683917 13537 leader_election.cc:288] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
I20260926 07:36:08.698864 12740 catalog_manager.cc:5495] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a reported cstate change: term changed from 0 to 1, leader changed from <none> to 16209ececd2d420588b390f95db9b92a (127.10.171.2). New cstate: current_term: 1 leader_uuid: "16209ececd2d420588b390f95db9b92a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } health_report { overall_health: UNKNOWN } } }
I20260926 07:36:08.704860 13525 ts_tablet_manager.cc:1918] T 932e147b05fe4e1b95942182562feffe P a6ff3db459444683972a22263d29009b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:08.705265 13525 log.cc:1196] T 932e147b05fe4e1b95942182562feffe P a6ff3db459444683972a22263d29009b: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/932e147b05fe4e1b95942182562feffe
I20260926 07:36:08.705340 12737 master_service.cc:946] ReplaceTablet: received request to replace tablet 8f0fde61e21444eaa091c9c3eb7ae8ec from {username='slave'} at 127.0.0.1:45538
I20260926 07:36:08.705961 13525 ts_tablet_manager.cc:1939] T 932e147b05fe4e1b95942182562feffe P a6ff3db459444683972a22263d29009b: Deleting consensus metadata
I20260926 07:36:08.708668 12731 catalog_manager.cc:4841] TS a6ff3db459444683972a22263d29009b (127.10.171.3:34663): tablet 932e147b05fe4e1b95942182562feffe (table default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]) successfully deleted
I20260926 07:36:08.708549 13526 ts_tablet_manager.cc:1918] T 9060d09b80e948a5b36b1004ae4f2321 P 095560781b2d4e58b3804fce75312b90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:08.715490 13263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37337c5502804878a77653b6d1964ea4" candidate_uuid: "095560781b2d4e58b3804fce75312b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16209ececd2d420588b390f95db9b92a" is_pre_election: true
I20260926 07:36:08.716064 13263 raft_consensus.cc:2454] T 37337c5502804878a77653b6d1964ea4 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 095560781b2d4e58b3804fce75312b90 in term 0.
I20260926 07:36:08.720340 13035 leader_election.cc:302] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a; no voters:
I20260926 07:36:08.720232 13526 log.cc:1196] T 9060d09b80e948a5b36b1004ae4f2321 P 095560781b2d4e58b3804fce75312b90: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/9060d09b80e948a5b36b1004ae4f2321
I20260926 07:36:08.721374 13537 raft_consensus.cc:2781] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:36:08.721733 13526 ts_tablet_manager.cc:1939] T 9060d09b80e948a5b36b1004ae4f2321 P 095560781b2d4e58b3804fce75312b90: Deleting consensus metadata
I20260926 07:36:08.721711 13537 raft_consensus.cc:482] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:36:08.722100 13537 raft_consensus.cc:3037] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:36:08.725502 13526 tablet_replica.cc:330] T 932e147b05fe4e1b95942182562feffe P 095560781b2d4e58b3804fce75312b90: stopping tablet replica
I20260926 07:36:08.726257 13526 raft_consensus.cc:2229] T 932e147b05fe4e1b95942182562feffe P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:08.726258 12731 catalog_manager.cc:4841] TS 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): tablet 9060d09b80e948a5b36b1004ae4f2321 (table default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]) successfully deleted
I20260926 07:36:08.726687 13526 raft_consensus.cc:2258] T 932e147b05fe4e1b95942182562feffe P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:08.730037 13537 raft_consensus.cc:504] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.728775 13526 ts_tablet_manager.cc:1905] T 932e147b05fe4e1b95942182562feffe P 095560781b2d4e58b3804fce75312b90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:08.733975 13263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37337c5502804878a77653b6d1964ea4" candidate_uuid: "095560781b2d4e58b3804fce75312b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16209ececd2d420588b390f95db9b92a"
I20260926 07:36:08.734616 13263 raft_consensus.cc:3037] T 37337c5502804878a77653b6d1964ea4 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Advancing to term 1
@ 0x7efc2e374dc7 kudu::Thread::SuperviseThread() at ??:0
@ 0x4230f0 __tsan_thread_start_func at ??:?
I20260926 07:36:08.738005 12737 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51342b641d47457d84f48fcbb6c48375: Sending DeleteTablet for 3 replicas of tablet 8f0fde61e21444eaa091c9c3eb7ae8ec
I20260926 07:36:08.741462 13263 raft_consensus.cc:2454] T 37337c5502804878a77653b6d1964ea4 P 16209ececd2d420588b390f95db9b92a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 095560781b2d4e58b3804fce75312b90 in term 1.
I20260926 07:36:08.742494 13035 leader_election.cc:302] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a; no voters:
I20260926 07:36:08.746584 12738 master_service.cc:946] ReplaceTablet: received request to replace tablet 932e147b05fe4e1b95942182562feffe from {username='slave'} at 127.0.0.1:45538
I20260926 07:36:08.747213 13244 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f0fde61e21444eaa091c9c3eb7ae8ec with delete_type TABLET_DATA_DELETED (replaced by tablet b7a83a4b4cbb4490b44acc66e4676c18) from {username='slave'} at 127.0.0.1:53506
I20260926 07:36:08.747881 13524 tablet_replica.cc:330] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 16209ececd2d420588b390f95db9b92a: stopping tablet replica
I20260926 07:36:08.748704 13524 raft_consensus.cc:2229] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:08.749181 13524 raft_consensus.cc:2258] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:08.749310 13526 ts_tablet_manager.cc:1918] T 932e147b05fe4e1b95942182562feffe P 095560781b2d4e58b3804fce75312b90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:08.749624 12737 catalog_manager.cc:6565] ReplaceTablet: tablet 8f0fde61e21444eaa091c9c3eb7ae8ec deleted and replaced by tablet b7a83a4b4cbb4490b44acc66e4676c18
I20260926 07:36:08.749603 13087 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f0fde61e21444eaa091c9c3eb7ae8ec with delete_type TABLET_DATA_DELETED (replaced by tablet b7a83a4b4cbb4490b44acc66e4676c18) from {username='slave'} at 127.0.0.1:34588
I20260926 07:36:08.749660 13526 log.cc:1196] T 932e147b05fe4e1b95942182562feffe P 095560781b2d4e58b3804fce75312b90: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/932e147b05fe4e1b95942182562feffe
I20260926 07:36:08.751032 13394 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f0fde61e21444eaa091c9c3eb7ae8ec with delete_type TABLET_DATA_DELETED (replaced by tablet b7a83a4b4cbb4490b44acc66e4676c18) from {username='slave'} at 127.0.0.1:45090
I20260926 07:36:08.751405 13524 ts_tablet_manager.cc:1905] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 16209ececd2d420588b390f95db9b92a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:08.751740 13525 tablet_replica.cc:330] T 8f0fde61e21444eaa091c9c3eb7ae8ec P a6ff3db459444683972a22263d29009b: stopping tablet replica
I20260926 07:36:08.752475 13525 raft_consensus.cc:2229] T 8f0fde61e21444eaa091c9c3eb7ae8ec P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:08.752933 13525 raft_consensus.cc:2258] T 8f0fde61e21444eaa091c9c3eb7ae8ec P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:08.755730 13525 ts_tablet_manager.cc:1905] T 8f0fde61e21444eaa091c9c3eb7ae8ec P a6ff3db459444683972a22263d29009b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:08.773331 13526 ts_tablet_manager.cc:1939] T 932e147b05fe4e1b95942182562feffe P 095560781b2d4e58b3804fce75312b90: Deleting consensus metadata
I20260926 07:36:08.768364 13537 leader_election.cc:288] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [CANDIDATE]: Term 1 election: Requested vote from peers 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
I20260926 07:36:08.780171 13526 tablet_replica.cc:330] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 095560781b2d4e58b3804fce75312b90: stopping tablet replica
I20260926 07:36:08.780452 13537 raft_consensus.cc:2781] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:36:08.780671 12731 catalog_manager.cc:4841] TS 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): tablet 932e147b05fe4e1b95942182562feffe (table default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]) successfully deleted
I20260926 07:36:08.780958 13537 raft_consensus.cc:686] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [term 1 LEADER]: Becoming Leader. State: Replica: 095560781b2d4e58b3804fce75312b90, State: Running, Role: LEADER
@ 0x7efc2f9e26db start_thread at ??:0
I20260926 07:36:08.782047 13537 consensus_queue.cc:237] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.784206 13525 ts_tablet_manager.cc:1918] T 8f0fde61e21444eaa091c9c3eb7ae8ec P a6ff3db459444683972a22263d29009b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:08.784554 13525 log.cc:1196] T 8f0fde61e21444eaa091c9c3eb7ae8ec P a6ff3db459444683972a22263d29009b: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/8f0fde61e21444eaa091c9c3eb7ae8ec
I20260926 07:36:08.785228 13525 ts_tablet_manager.cc:1939] T 8f0fde61e21444eaa091c9c3eb7ae8ec P a6ff3db459444683972a22263d29009b: Deleting consensus metadata
I20260926 07:36:08.780977 13526 raft_consensus.cc:2229] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:08.786736 13526 raft_consensus.cc:2258] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:08.786890 12731 catalog_manager.cc:4841] TS a6ff3db459444683972a22263d29009b (127.10.171.3:34663): tablet 8f0fde61e21444eaa091c9c3eb7ae8ec (table default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]) successfully deleted
I20260926 07:36:08.788918 13526 ts_tablet_manager.cc:1905] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 095560781b2d4e58b3804fce75312b90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:08.792872 13394 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f0fde61e21444eaa091c9c3eb7ae8ec with delete_type TABLET_DATA_DELETED (replaced by tablet b7a83a4b4cbb4490b44acc66e4676c18) from {username='slave'} at 127.0.0.1:45090
W20260926 07:36:08.797358 12731 catalog_manager.cc:4808] TS a6ff3db459444683972a22263d29009b (127.10.171.3:34663): delete failed for tablet 8f0fde61e21444eaa091c9c3eb7ae8ec because the tablet was not found. No further retry: Not found: Tablet not found: 8f0fde61e21444eaa091c9c3eb7ae8ec
I20260926 07:36:08.802989 13519 tablet_bootstrap.cc:492] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b: Bootstrap starting.
I20260926 07:36:08.803094 13526 ts_tablet_manager.cc:1918] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 095560781b2d4e58b3804fce75312b90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:08.803407 13526 log.cc:1196] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 095560781b2d4e58b3804fce75312b90: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/8f0fde61e21444eaa091c9c3eb7ae8ec
I20260926 07:36:08.804235 13526 ts_tablet_manager.cc:1939] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 095560781b2d4e58b3804fce75312b90: Deleting consensus metadata
I20260926 07:36:08.806689 12731 catalog_manager.cc:4841] TS 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): tablet 8f0fde61e21444eaa091c9c3eb7ae8ec (table default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]) successfully deleted
I20260926 07:36:08.807819 13414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37337c5502804878a77653b6d1964ea4" candidate_uuid: "095560781b2d4e58b3804fce75312b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ff3db459444683972a22263d29009b" is_pre_election: true
I20260926 07:36:08.808033 13413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37337c5502804878a77653b6d1964ea4" candidate_uuid: "095560781b2d4e58b3804fce75312b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ff3db459444683972a22263d29009b"
W20260926 07:36:08.809378 13034 leader_election.cc:341] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Illegal state: must be running to vote when last-logged opid is not known
W20260926 07:36:08.810313 13034 leader_election.cc:341] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:36:08.815064 13524 ts_tablet_manager.cc:1918] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 16209ececd2d420588b390f95db9b92a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:08.815443 13524 log.cc:1196] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 16209ececd2d420588b390f95db9b92a: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/8f0fde61e21444eaa091c9c3eb7ae8ec
@ 0x7efc2ae6861f clone at ??:0
I20260926 07:36:08.831611 13524 ts_tablet_manager.cc:1939] T 8f0fde61e21444eaa091c9c3eb7ae8ec P 16209ececd2d420588b390f95db9b92a: Deleting consensus metadata
I20260926 07:36:08.833293 13519 tablet_bootstrap.cc:654] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:08.851231 13519 tablet_bootstrap.cc:492] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b: No bootstrap required, opened a new log
I20260926 07:36:08.851617 13519 ts_tablet_manager.cc:1397] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b: Time spent bootstrapping tablet: real 0.049s user 0.008s sys 0.001s
I20260926 07:36:08.853616 13519 raft_consensus.cc:348] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.854214 13519 raft_consensus.cc:374] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:08.854418 13519 raft_consensus.cc:729] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ff3db459444683972a22263d29009b, State: Initialized, Role: FOLLOWER
I20260926 07:36:08.855105 13519 consensus_queue.cc:260] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:08.862901 13519 ts_tablet_manager.cc:1428] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b: Time spent starting tablet: real 0.011s user 0.002s sys 0.004s
W20260926 07:36:09.021981 13530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:36:09.022656 13530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:36:09.023366 13530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:36:09.067162 13530 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260926 07:36:09.067536 13530 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260926 07:36:09.067890 13530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:36:09.068271 13530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:36:09.068621 13530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:36:09.068960 13530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:36:09.126343 13530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.171.20:42033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:35621
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.171.60:45615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.171.60
--webserver_port=40625
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:55:20 UTC on ccycloud.quasar-qwhbmo.root.comops.site
build id 4673737
TSAN enabled
I20260926 07:36:09.127650 13530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:36:09.129426 13530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:36:09.149024 13596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:36:09.152493 13473 connection.cc:535] client connection to 127.10.171.62:36693 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:36:09.153223 13349 connection.cc:535] client connection to 127.10.171.62:36693 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:36:09.153225 13036 connection.cc:535] client connection to 127.10.171.62:36693 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:36:09.154630 13473 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.171.62:36693)
W20260926 07:36:09.155553 13473 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.10.171.62:36693)
W20260926 07:36:09.156595 13527 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 07:36:09.154820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.171.62:36693 (local address 127.10.171.3:50293)
0926 07:36:09.155339 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0926 07:36:09.155532 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":89}
W20260926 07:36:09.156621 13195 connection.cc:535] server connection from 127.0.0.1:53506 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:36:09.162223 13473 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.10.171.62:36693)
W20260926 07:36:09.163051 13473 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.10.171.62:36693)
I20260926 07:36:09.268692 13602 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 51342b641d47457d84f48fcbb6c48375)
I20260926 07:36:09.269135 13602 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:09.275777 13602 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:36:09.280731 12808 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:36:09.282706 12807 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:36:09.283305 12807 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:09.284274 13602 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260926 07:36:09.519327 13606 raft_consensus.cc:482] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:09.519909 13606 raft_consensus.cc:504] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:09.523773 13606 leader_election.cc:288] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
I20260926 07:36:09.523986 13263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37337c5502804878a77653b6d1964ea4" candidate_uuid: "a6ff3db459444683972a22263d29009b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16209ececd2d420588b390f95db9b92a" is_pre_election: true
I20260926 07:36:09.524820 13263 raft_consensus.cc:2379] T 37337c5502804878a77653b6d1964ea4 P 16209ececd2d420588b390f95db9b92a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a6ff3db459444683972a22263d29009b in current term 1: Already voted for candidate 095560781b2d4e58b3804fce75312b90 in this term.
I20260926 07:36:09.525497 13108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37337c5502804878a77653b6d1964ea4" candidate_uuid: "a6ff3db459444683972a22263d29009b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "095560781b2d4e58b3804fce75312b90" is_pre_election: true
I20260926 07:36:09.527204 13347 leader_election.cc:302] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:36:09.528417 13606 raft_consensus.cc:3037] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:36:09.535368 13606 raft_consensus.cc:2726] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:36:09.589951 13607 consensus_queue.cc:1035] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:36:09.649479 13608 consensus_queue.cc:1035] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:36:09.658519 13609 consensus_queue.cc:1035] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [LEADER]: Connected to new peer: Peer: permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 07:36:09.739073 13609 consensus_queue.cc:1035] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 07:36:09.981267 13626 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 51342b641d47457d84f48fcbb6c48375)
I20260926 07:36:09.981679 13626 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:09.983249 13626 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:36:09.989827 12808 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:36:09.990350 12807 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:36:09.990973 12807 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:09.991958 13626 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260926 07:36:09.156682 13593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:36:09.149468 13594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 07:36:10.543545 13595 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260926 07:36:10.543599 13530 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260926 07:36:10.548574 13530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:36:10.551775 13530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:36:10.553220 13530 hybrid_clock.cc:648] HybridClock initialized: now 1790408170553146 us; error 89 us; skew 500 ppm
I20260926 07:36:10.554070 13530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:36:10.561718 13530 webserver.cc:460] Webserver started at http://127.10.171.60:40625/ using document root <none> and password file <none>
I20260926 07:36:10.562718 13530 fs_manager.cc:359] Metadata directory not provided
I20260926 07:36:10.562949 13530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:36:10.571978 13530 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 07:36:10.576828 13641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:36:10.579160 13530 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.004s
I20260926 07:36:10.579596 13530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/data,/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/wal
uuid: "0a6ef379bc314e798568d59eba3a3a91"
format_stamp: "Formatted at 2026-09-26 07:35:58 on dist-test-slave-3mmx"
I20260926 07:36:10.582161 13530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:36:10.665463 13530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:36:10.666954 13530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:36:10.667363 13530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:36:10.701620 13663 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 51342b641d47457d84f48fcbb6c48375)
I20260926 07:36:10.702201 13663 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:10.704507 13663 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:36:10.711476 12808 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:36:10.713230 12807 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:36:10.713579 12807 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:10.714170 13663 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260926 07:36:10.746785 13530 rpc_server.cc:307] RPC server started. Bound to: 127.10.171.60:45615
I20260926 07:36:10.746927 13696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.171.60:45615 every 8 connection(s)
I20260926 07:36:10.750166 13530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-2/data/info.pb
I20260926 07:36:10.759294 10924 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu as pid 13530
I20260926 07:36:10.764638 13697 sys_catalog.cc:263] Verifying existing consensus state
I20260926 07:36:10.775461 13697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: Bootstrap starting.
I20260926 07:36:10.787273 13460 heartbeater.cc:347] Connected to a master server at 127.10.171.60:45615
I20260926 07:36:10.787804 13460 heartbeater.cc:503] Master 127.10.171.60:45615 was elected leader, sending a full tablet report...
I20260926 07:36:10.792753 13697 log.cc:824] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: Log is configured to *not* fsync() on all Append() calls
I20260926 07:36:10.814991 13697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 07:36:10.815743 13697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: Bootstrap complete.
I20260926 07:36:10.835976 13697 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:10.838778 13697 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a6ef379bc314e798568d59eba3a3a91, State: Initialized, Role: FOLLOWER
I20260926 07:36:10.839634 13697 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:10.842690 13702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:10.843544 13702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:36:10.844247 13697 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:36:10.889652 13717 catalog_manager.cc:1259] Loaded cluster ID: 6e2e15d6975946d1a4009a3693fa16a8
I20260926 07:36:10.890033 13717 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: loading cluster ID for follower catalog manager: success
I20260926 07:36:10.898317 13717 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: acquiring CA information for follower catalog manager: success
I20260926 07:36:10.904382 13717 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 07:36:10.888886 13530 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:36:10.923377 13530 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:36:10.931882 13154 heartbeater.cc:347] Connected to a master server at 127.10.171.60:45615
I20260926 07:36:10.940615 13658 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "095560781b2d4e58b3804fce75312b90" instance_seqno: 1790408162365667) as {username='slave'} at 127.10.171.1:51285; Asking this server to re-register.
I20260926 07:36:10.942258 13154 heartbeater.cc:464] Registering TS with master...
I20260926 07:36:10.946286 13658 ts_manager.cc:194] Registered new tserver with Master: 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253)
I20260926 07:36:11.042363 13310 heartbeater.cc:347] Connected to a master server at 127.10.171.60:45615
I20260926 07:36:11.046161 13658 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "16209ececd2d420588b390f95db9b92a" instance_seqno: 1790408164464241) as {username='slave'} at 127.10.171.2:52217; Asking this server to re-register.
I20260926 07:36:11.047456 13310 heartbeater.cc:464] Registering TS with master...
I20260926 07:36:11.050470 13658 ts_manager.cc:194] Registered new tserver with Master: 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
I20260926 07:36:11.241858 13724 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 51342b641d47457d84f48fcbb6c48375)
I20260926 07:36:11.242255 13724 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:11.243819 13724 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:36:11.250489 12808 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:36:11.256641 13672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7726b3d3f73544318efe4a3be7a63f70" candidate_term: 5 candidate_status { last_received { term: 4 index: 28 } } ignore_live_leader: false dest_uuid: "0a6ef379bc314e798568d59eba3a3a91" is_pre_election: true
I20260926 07:36:11.257453 13672 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7726b3d3f73544318efe4a3be7a63f70 in term 3.
I20260926 07:36:11.258441 12807 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a6ef379bc314e798568d59eba3a3a91, 7726b3d3f73544318efe4a3be7a63f70; no voters: 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:11.259058 13724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260926 07:36:11.259373 13724 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 51342b641d47457d84f48fcbb6c48375)
I20260926 07:36:11.259634 13724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 4 FOLLOWER]: Advancing to term 5
I20260926 07:36:11.264979 13724 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:11.266741 13724 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 5 election: Requested vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
I20260926 07:36:11.268090 13672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7726b3d3f73544318efe4a3be7a63f70" candidate_term: 5 candidate_status { last_received { term: 4 index: 28 } } ignore_live_leader: false dest_uuid: "0a6ef379bc314e798568d59eba3a3a91"
I20260926 07:36:11.268656 13672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 3 FOLLOWER]: Advancing to term 5
W20260926 07:36:11.270988 12808 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:36:11.278275 13672 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7726b3d3f73544318efe4a3be7a63f70 in term 5.
I20260926 07:36:11.280802 12807 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a6ef379bc314e798568d59eba3a3a91, 7726b3d3f73544318efe4a3be7a63f70; no voters: 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:11.281968 13724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 5 FOLLOWER]: Leader election won for term 5
I20260926 07:36:11.284225 13724 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 5 LEADER]: Becoming Leader. State: Replica: 7726b3d3f73544318efe4a3be7a63f70, State: Running, Role: LEADER
I20260926 07:36:11.285138 13724 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 28, Committed index: 28, Last appended: 4.28, Last appended by leader: 28, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:11.292441 13460 heartbeater.cc:503] Master 127.10.171.60:45615 was elected leader, sending a full tablet report...
I20260926 07:36:11.292611 13728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7726b3d3f73544318efe4a3be7a63f70. Latest consensus state: current_term: 5 leader_uuid: "7726b3d3f73544318efe4a3be7a63f70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:11.293334 13728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: This master's current role is: LEADER
I20260926 07:36:11.295909 13730 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:36:11.296394 13658 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "a6ff3db459444683972a22263d29009b" instance_seqno: 1790408166437023) as {username='slave'} at 127.10.171.3:49563; Asking this server to re-register.
I20260926 07:36:11.297947 13460 heartbeater.cc:464] Registering TS with master...
I20260926 07:36:11.301389 13658 ts_manager.cc:194] Registered new tserver with Master: a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
I20260926 07:36:11.305083 13730 catalog_manager.cc:669] Loaded metadata for table default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]
I20260926 07:36:11.306934 13730 catalog_manager.cc:669] Loaded metadata for table default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]
I20260926 07:36:11.308161 13730 catalog_manager.cc:669] Loaded metadata for table default.table_5280e066b7404013a7c7bdfe8e9bc18b [id=e093563ca611494caed51759972753c6]
I20260926 07:36:11.313568 13730 tablet_loader.cc:96] loaded metadata for tablet 37337c5502804878a77653b6d1964ea4 (table default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6])
I20260926 07:36:11.314584 13730 tablet_loader.cc:96] loaded metadata for tablet 758385269fe544bfa148f596a1649143 (table default.table_5280e066b7404013a7c7bdfe8e9bc18b [id=e093563ca611494caed51759972753c6])
I20260926 07:36:11.317077 13730 tablet_loader.cc:96] loaded metadata for tablet b7a83a4b4cbb4490b44acc66e4676c18 (table default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f])
I20260926 07:36:11.317905 13730 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:36:11.321018 13730 catalog_manager.cc:1259] Loaded cluster ID: 6e2e15d6975946d1a4009a3693fa16a8
I20260926 07:36:11.321228 13730 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:36:11.325050 13730 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:36:11.328052 13730 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70: Loaded TSK: 0
I20260926 07:36:11.329038 13730 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:36:11.332131 13730 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260926 07:36:11.334924 13730 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260926 07:36:11.363322 12820 master_service.cc:946] ReplaceTablet: received request to replace tablet 37337c5502804878a77653b6d1964ea4 from {username='slave'} at 127.0.0.1:33658
I20260926 07:36:11.367066 12814 master_service.cc:946] ReplaceTablet: received request to replace tablet 37337c5502804878a77653b6d1964ea4 from {username='slave'} at 127.0.0.1:33658
I20260926 07:36:11.366921 12815 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33658:
table { table_name: "default.Table_708d07ef404b42629d7c958b81980ca1" } new_table_name: "default.Table_9fbcc2f37250434e80c28330fb80f80b" modify_external_catalogs: true
I20260926 07:36:11.364629 12818 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33658:
table { table_name: "default.Table_5280e066b7404013a7c7bdfe8e9bc18b" } new_table_name: "default.Table_b2944ae24c6f47d4a1c3cb780994e4fe" modify_external_catalogs: true
I20260926 07:36:11.362151 12819 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33658:
table { table_name: "default.Table_b9246fcaf5ed4103b8c7835af689efab" } modify_external_catalogs: true
I20260926 07:36:11.381307 12819 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33658:
table { table_name: "default.Table_b9246fcaf5ed4103b8c7835af689efab" } modify_external_catalogs: true
I20260926 07:36:11.369025 12813 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33658:
name: "default.Table_6d097d04d5e6412d9f5354f5c40d580f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:36:11.366091 12821 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33658:
name: "default.Table_5af6599ea22546fdb4c598d6a124f28d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:36:11.365440 12817 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33658:
name: "default.Table_daf8f6d0f2c6493d879b9a3bfbd1cfad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:36:11.363735 12816 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33658:
name: "default.Table_8019d82e72b54af899515d80cb8aa33e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260926 07:36:11.388660 13006 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_708d07ef404b42629d7c958b81980ca1
*** Check failure stack trace: ***
I20260926 07:36:11.395231 13672 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 5 FOLLOWER]: Refusing update from remote peer 7726b3d3f73544318efe4a3be7a63f70: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 30. (index mismatch)
I20260926 07:36:11.397732 13724 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 29, Last known committed idx: 6, Time since last communication: 0.000s
W20260926 07:36:11.401784 12808 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 -> Peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Couldn't send request to peer 51342b641d47457d84f48fcbb6c48375. Status: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
@ 0x7f39c4b539a0 google::LogMessage::Flush() at ??:0
@ 0x7f39c4b5370f google::LogMessage::~LogMessage() at ??:0
@ 0x7f39cfbeeaea kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260926 07:36:11.494452 13733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7726b3d3f73544318efe4a3be7a63f70. Latest consensus state: current_term: 5 leader_uuid: "7726b3d3f73544318efe4a3be7a63f70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:11.495239 13733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:36:11.500190 13738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "7726b3d3f73544318efe4a3be7a63f70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:11.501068 13738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f39cfbee33c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260926 07:36:11.519548 13738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "7726b3d3f73544318efe4a3be7a63f70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:11.520373 13738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:36:11.525647 13728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "7726b3d3f73544318efe4a3be7a63f70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:11.526435 13728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: This master's current role is: LEADER
I20260926 07:36:11.567663 12820 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70: Sending DeleteTablet for 3 replicas of tablet 37337c5502804878a77653b6d1964ea4
@ 0x7f39cfbf1fd2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260926 07:36:11.573293 12820 catalog_manager.cc:6565] ReplaceTablet: tablet 37337c5502804878a77653b6d1964ea4 deleted and replaced by tablet 385ef5d71f3a49c8b5057525a388230d
I20260926 07:36:11.586309 13244 tablet_service.cc:1467] Processing CreateTablet for tablet b7a83a4b4cbb4490b44acc66e4676c18 (DEFAULT_TABLE table=default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:11.587360 13244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7a83a4b4cbb4490b44acc66e4676c18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:11.588370 13243 tablet_service.cc:1514] Processing DeleteTablet for tablet 37337c5502804878a77653b6d1964ea4 with delete_type TABLET_DATA_DELETED (replaced by tablet 385ef5d71f3a49c8b5057525a388230d) from {username='slave'} at 127.0.0.1:51478
I20260926 07:36:11.594815 13087 tablet_service.cc:1467] Processing CreateTablet for tablet b7a83a4b4cbb4490b44acc66e4676c18 (DEFAULT_TABLE table=default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]), partition=RANGE (key) PARTITION UNBOUNDED
@ 0x7f39cfbf1f8a _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260926 07:36:11.596100 13087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7a83a4b4cbb4490b44acc66e4676c18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:11.596449 13088 tablet_service.cc:1514] Processing DeleteTablet for tablet 37337c5502804878a77653b6d1964ea4 with delete_type TABLET_DATA_DELETED (replaced by tablet 385ef5d71f3a49c8b5057525a388230d) from {username='slave'} at 127.0.0.1:51078
I20260926 07:36:11.620479 13394 tablet_service.cc:1467] Processing CreateTablet for tablet b7a83a4b4cbb4490b44acc66e4676c18 (DEFAULT_TABLE table=default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:11.621016 13752 tablet_replica.cc:330] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90: stopping tablet replica
I20260926 07:36:11.621848 13394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7a83a4b4cbb4490b44acc66e4676c18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:11.621731 13752 raft_consensus.cc:2229] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:36:11.622861 13752 raft_consensus.cc:2258] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:11.624045 13393 tablet_service.cc:1514] Processing DeleteTablet for tablet 37337c5502804878a77653b6d1964ea4 with delete_type TABLET_DATA_DELETED (replaced by tablet 385ef5d71f3a49c8b5057525a388230d) from {username='slave'} at 127.0.0.1:42394
I20260926 07:36:11.624994 13752 ts_tablet_manager.cc:1905] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:11.640163 13088 tablet_service.cc:1467] Processing CreateTablet for tablet 385ef5d71f3a49c8b5057525a388230d (DEFAULT_TABLE table=default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:11.641309 13088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 385ef5d71f3a49c8b5057525a388230d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:11.647586 13392 tablet_service.cc:1467] Processing CreateTablet for tablet 385ef5d71f3a49c8b5057525a388230d (DEFAULT_TABLE table=default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:11.647650 13242 tablet_service.cc:1467] Processing CreateTablet for tablet 385ef5d71f3a49c8b5057525a388230d (DEFAULT_TABLE table=default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:11.648820 13392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 385ef5d71f3a49c8b5057525a388230d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:11.648814 13242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 385ef5d71f3a49c8b5057525a388230d. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f39cfbf1f1a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
I20260926 07:36:11.668308 13751 tablet_replica.cc:330] T 37337c5502804878a77653b6d1964ea4 P 16209ececd2d420588b390f95db9b92a: stopping tablet replica
I20260926 07:36:11.669113 13751 raft_consensus.cc:2229] T 37337c5502804878a77653b6d1964ea4 P 16209ececd2d420588b390f95db9b92a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:11.669618 13751 raft_consensus.cc:2258] T 37337c5502804878a77653b6d1964ea4 P 16209ececd2d420588b390f95db9b92a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:11.670845 13752 ts_tablet_manager.cc:1918] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:36:11.671042 13751 ts_tablet_manager.cc:1905] T 37337c5502804878a77653b6d1964ea4 P 16209ececd2d420588b390f95db9b92a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:11.671237 13752 log.cc:1196] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/37337c5502804878a77653b6d1964ea4
I20260926 07:36:11.672086 13752 ts_tablet_manager.cc:1939] T 37337c5502804878a77653b6d1964ea4 P 095560781b2d4e58b3804fce75312b90: Deleting consensus metadata
I20260926 07:36:11.674834 12806 catalog_manager.cc:4841] TS 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): tablet 37337c5502804878a77653b6d1964ea4 (table default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]) successfully deleted
I20260926 07:36:11.677682 13735 mvcc.cc:205] Tried to move back new op lower bound from 7333511869989978112 to 7333511851811454976. Current Snapshot: MvccSnapshot[applied={T|T < 7333511869989978112 or (T in {7333511869989978112})}]
I20260926 07:36:11.681969 13755 tablet_bootstrap.cc:492] T b7a83a4b4cbb4490b44acc66e4676c18 P 095560781b2d4e58b3804fce75312b90: Bootstrap starting.
I20260926 07:36:11.688462 13755 tablet_bootstrap.cc:654] T b7a83a4b4cbb4490b44acc66e4676c18 P 095560781b2d4e58b3804fce75312b90: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:11.698567 13754 tablet_replica.cc:330] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b: stopping tablet replica
I20260926 07:36:11.699412 13754 raft_consensus.cc:2229] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:11.699963 13754 raft_consensus.cc:2258] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b [term 1 FOLLOWER]: Raft consensus is shut down!
@ 0x7f39cfbf1ee2 std::__1::__function::__alloc_func<>::operator()() at ??:0
I20260926 07:36:11.713603 13753 tablet_bootstrap.cc:492] T b7a83a4b4cbb4490b44acc66e4676c18 P 16209ececd2d420588b390f95db9b92a: Bootstrap starting.
I20260926 07:36:11.720764 13754 ts_tablet_manager.cc:1905] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f39cfbf11de std::__1::__function::__func<>::operator()() at ??:0
@ 0x580785 std::__1::__function::__value_func<>::operator()() at ??:?
@ 0x5805ba std::__1::function<>::operator()() at ??:?
I20260926 07:36:11.761428 13759 tablet_bootstrap.cc:492] T 385ef5d71f3a49c8b5057525a388230d P a6ff3db459444683972a22263d29009b: Bootstrap starting.
@ 0x7f39c6d04dc7 kudu::Thread::SuperviseThread() at ??:0
@ 0x4230f0 __tsan_thread_start_func at ??:?
I20260926 07:36:11.765570 13755 tablet_bootstrap.cc:492] T b7a83a4b4cbb4490b44acc66e4676c18 P 095560781b2d4e58b3804fce75312b90: No bootstrap required, opened a new log
I20260926 07:36:11.766012 13755 ts_tablet_manager.cc:1397] T b7a83a4b4cbb4490b44acc66e4676c18 P 095560781b2d4e58b3804fce75312b90: Time spent bootstrapping tablet: real 0.084s user 0.011s sys 0.045s
I20260926 07:36:11.768708 13755 raft_consensus.cc:348] T b7a83a4b4cbb4490b44acc66e4676c18 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:11.769513 13755 raft_consensus.cc:374] T b7a83a4b4cbb4490b44acc66e4676c18 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:11.769788 13755 raft_consensus.cc:729] T b7a83a4b4cbb4490b44acc66e4676c18 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 095560781b2d4e58b3804fce75312b90, State: Initialized, Role: FOLLOWER
I20260926 07:36:11.770437 13755 consensus_queue.cc:260] T b7a83a4b4cbb4490b44acc66e4676c18 P 095560781b2d4e58b3804fce75312b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
@ 0x7f39c83726db start_thread at ??:0
I20260926 07:36:11.783581 13153 heartbeater.cc:503] Master 127.10.171.61:45551 was elected leader, sending a full tablet report...
I20260926 07:36:11.785166 12814 master_service.cc:946] ReplaceTablet: received request to replace tablet 758385269fe544bfa148f596a1649143 from {username='slave'} at 127.0.0.1:33658
I20260926 07:36:11.785532 12820 master_service.cc:946] ReplaceTablet: received request to replace tablet 385ef5d71f3a49c8b5057525a388230d from {username='slave'} at 127.0.0.1:33658
I20260926 07:36:11.792970 13755 ts_tablet_manager.cc:1428] T b7a83a4b4cbb4490b44acc66e4676c18 P 095560781b2d4e58b3804fce75312b90: Time spent starting tablet: real 0.027s user 0.004s sys 0.004s
I20260926 07:36:11.793895 13755 tablet_bootstrap.cc:492] T 385ef5d71f3a49c8b5057525a388230d P 095560781b2d4e58b3804fce75312b90: Bootstrap starting.
I20260926 07:36:11.814489 12814 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70: Sending DeleteTablet for 3 replicas of tablet 758385269fe544bfa148f596a1649143
I20260926 07:36:11.815685 13755 tablet_bootstrap.cc:654] T 385ef5d71f3a49c8b5057525a388230d P 095560781b2d4e58b3804fce75312b90: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:11.816340 13087 tablet_service.cc:1514] Processing DeleteTablet for tablet 758385269fe544bfa148f596a1649143 with delete_type TABLET_DATA_DELETED (replaced by tablet 23b91b8716e5405e8b3aad7df68183fc) from {username='slave'} at 127.0.0.1:51078
I20260926 07:36:11.817102 13752 tablet_replica.cc:330] T 758385269fe544bfa148f596a1649143 P 095560781b2d4e58b3804fce75312b90: stopping tablet replica
I20260926 07:36:11.817821 13752 raft_consensus.cc:2229] T 758385269fe544bfa148f596a1649143 P 095560781b2d4e58b3804fce75312b90 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:11.818295 13752 raft_consensus.cc:2258] T 758385269fe544bfa148f596a1649143 P 095560781b2d4e58b3804fce75312b90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:11.821444 13752 ts_tablet_manager.cc:1905] T 758385269fe544bfa148f596a1649143 P 095560781b2d4e58b3804fce75312b90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:11.825906 12814 catalog_manager.cc:6565] ReplaceTablet: tablet 758385269fe544bfa148f596a1649143 deleted and replaced by tablet 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:36:11.827518 13755 tablet_bootstrap.cc:492] T 385ef5d71f3a49c8b5057525a388230d P 095560781b2d4e58b3804fce75312b90: No bootstrap required, opened a new log
I20260926 07:36:11.827893 13755 ts_tablet_manager.cc:1397] T 385ef5d71f3a49c8b5057525a388230d P 095560781b2d4e58b3804fce75312b90: Time spent bootstrapping tablet: real 0.034s user 0.011s sys 0.004s
I20260926 07:36:11.828312 13244 tablet_service.cc:1514] Processing DeleteTablet for tablet 758385269fe544bfa148f596a1649143 with delete_type TABLET_DATA_DELETED (replaced by tablet 23b91b8716e5405e8b3aad7df68183fc) from {username='slave'} at 127.0.0.1:51478
I20260926 07:36:11.829010 13753 tablet_bootstrap.cc:700] T b7a83a4b4cbb4490b44acc66e4676c18 P 16209ececd2d420588b390f95db9b92a: Time spent opening tablet: real 0.110s user 0.003s sys 0.000s
I20260926 07:36:11.829598 13753 tablet_bootstrap.cc:654] T b7a83a4b4cbb4490b44acc66e4676c18 P 16209ececd2d420588b390f95db9b92a: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:11.831471 13755 raft_consensus.cc:348] T 385ef5d71f3a49c8b5057525a388230d P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:11.832235 13755 raft_consensus.cc:374] T 385ef5d71f3a49c8b5057525a388230d P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:11.832509 13755 raft_consensus.cc:729] T 385ef5d71f3a49c8b5057525a388230d P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 095560781b2d4e58b3804fce75312b90, State: Initialized, Role: FOLLOWER
I20260926 07:36:11.832863 12820 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70: Sending DeleteTablet for 3 replicas of tablet 385ef5d71f3a49c8b5057525a388230d
I20260926 07:36:11.833221 13755 consensus_queue.cc:260] T 385ef5d71f3a49c8b5057525a388230d P 095560781b2d4e58b3804fce75312b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:11.834327 12820 catalog_manager.cc:6565] ReplaceTablet: tablet 385ef5d71f3a49c8b5057525a388230d deleted and replaced by tablet 3efbc3ccb32d46919864ad44f7cf630d
I20260926 07:36:11.835409 13755 ts_tablet_manager.cc:1428] T 385ef5d71f3a49c8b5057525a388230d P 095560781b2d4e58b3804fce75312b90: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260926 07:36:11.838934 13087 tablet_service.cc:1514] Processing DeleteTablet for tablet 385ef5d71f3a49c8b5057525a388230d with delete_type TABLET_DATA_DELETED (replaced by tablet 3efbc3ccb32d46919864ad44f7cf630d) from {username='slave'} at 127.0.0.1:51078
I20260926 07:36:11.845896 13244 tablet_service.cc:1514] Processing DeleteTablet for tablet 385ef5d71f3a49c8b5057525a388230d with delete_type TABLET_DATA_DELETED (replaced by tablet 3efbc3ccb32d46919864ad44f7cf630d) from {username='slave'} at 127.0.0.1:51478
@ 0x7f39c37f861f clone at ??:0
I20260926 07:36:11.851393 13753 tablet_bootstrap.cc:492] T b7a83a4b4cbb4490b44acc66e4676c18 P 16209ececd2d420588b390f95db9b92a: No bootstrap required, opened a new log
I20260926 07:36:11.853870 13759 tablet_bootstrap.cc:654] T 385ef5d71f3a49c8b5057525a388230d P a6ff3db459444683972a22263d29009b: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:11.855809 13751 ts_tablet_manager.cc:1918] T 37337c5502804878a77653b6d1964ea4 P 16209ececd2d420588b390f95db9b92a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:36:11.856274 13751 log.cc:1196] T 37337c5502804878a77653b6d1964ea4 P 16209ececd2d420588b390f95db9b92a: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/37337c5502804878a77653b6d1964ea4
I20260926 07:36:11.857089 13751 ts_tablet_manager.cc:1939] T 37337c5502804878a77653b6d1964ea4 P 16209ececd2d420588b390f95db9b92a: Deleting consensus metadata
I20260926 07:36:11.860203 13751 tablet_replica.cc:330] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a: stopping tablet replica
I20260926 07:36:11.860630 13753 ts_tablet_manager.cc:1397] T b7a83a4b4cbb4490b44acc66e4676c18 P 16209ececd2d420588b390f95db9b92a: Time spent bootstrapping tablet: real 0.147s user 0.012s sys 0.000s
I20260926 07:36:11.860895 13751 raft_consensus.cc:2229] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:36:11.861920 13751 raft_consensus.cc:2258] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:11.863270 13753 raft_consensus.cc:348] T b7a83a4b4cbb4490b44acc66e4676c18 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:11.865829 13759 tablet_bootstrap.cc:492] T 385ef5d71f3a49c8b5057525a388230d P a6ff3db459444683972a22263d29009b: No bootstrap required, opened a new log
I20260926 07:36:11.869678 13752 ts_tablet_manager.cc:1918] T 758385269fe544bfa148f596a1649143 P 095560781b2d4e58b3804fce75312b90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:36:11.870100 13752 log.cc:1196] T 758385269fe544bfa148f596a1649143 P 095560781b2d4e58b3804fce75312b90: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/758385269fe544bfa148f596a1649143
I20260926 07:36:11.870498 13753 raft_consensus.cc:374] T b7a83a4b4cbb4490b44acc66e4676c18 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:11.870904 13752 ts_tablet_manager.cc:1939] T 758385269fe544bfa148f596a1649143 P 095560781b2d4e58b3804fce75312b90: Deleting consensus metadata
I20260926 07:36:11.871230 13753 raft_consensus.cc:729] T b7a83a4b4cbb4490b44acc66e4676c18 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16209ececd2d420588b390f95db9b92a, State: Initialized, Role: FOLLOWER
I20260926 07:36:11.871803 13759 ts_tablet_manager.cc:1397] T 385ef5d71f3a49c8b5057525a388230d P a6ff3db459444683972a22263d29009b: Time spent bootstrapping tablet: real 0.111s user 0.012s sys 0.000s
I20260926 07:36:11.874356 13752 tablet_replica.cc:330] T 385ef5d71f3a49c8b5057525a388230d P 095560781b2d4e58b3804fce75312b90: stopping tablet replica
I20260926 07:36:11.863826 13751 ts_tablet_manager.cc:1905] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:11.875085 13752 raft_consensus.cc:2229] T 385ef5d71f3a49c8b5057525a388230d P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:11.876053 13752 raft_consensus.cc:2258] T 385ef5d71f3a49c8b5057525a388230d P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:11.874759 13392 tablet_service.cc:1514] Processing DeleteTablet for tablet 758385269fe544bfa148f596a1649143 with delete_type TABLET_DATA_DELETED (replaced by tablet 23b91b8716e5405e8b3aad7df68183fc) from {username='slave'} at 127.0.0.1:42394
I20260926 07:36:11.875633 13394 tablet_service.cc:1514] Processing DeleteTablet for tablet 385ef5d71f3a49c8b5057525a388230d with delete_type TABLET_DATA_DELETED (replaced by tablet 3efbc3ccb32d46919864ad44f7cf630d) from {username='slave'} at 127.0.0.1:42394
I20260926 07:36:11.876813 13753 consensus_queue.cc:260] T b7a83a4b4cbb4490b44acc66e4676c18 P 16209ececd2d420588b390f95db9b92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:11.878684 13752 ts_tablet_manager.cc:1905] T 385ef5d71f3a49c8b5057525a388230d P 095560781b2d4e58b3804fce75312b90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:11.885017 13309 heartbeater.cc:503] Master 127.10.171.61:45551 was elected leader, sending a full tablet report...
I20260926 07:36:11.885413 13753 ts_tablet_manager.cc:1428] T b7a83a4b4cbb4490b44acc66e4676c18 P 16209ececd2d420588b390f95db9b92a: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
I20260926 07:36:11.886240 13753 tablet_bootstrap.cc:492] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a: Bootstrap starting.
I20260926 07:36:11.869621 13754 ts_tablet_manager.cc:1918] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:36:11.888355 13754 log.cc:1196] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/37337c5502804878a77653b6d1964ea4
I20260926 07:36:11.890555 13754 ts_tablet_manager.cc:1939] T 37337c5502804878a77653b6d1964ea4 P a6ff3db459444683972a22263d29009b: Deleting consensus metadata
I20260926 07:36:11.892413 13759 raft_consensus.cc:348] T 385ef5d71f3a49c8b5057525a388230d P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:11.893167 13759 raft_consensus.cc:374] T 385ef5d71f3a49c8b5057525a388230d P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:11.893443 13759 raft_consensus.cc:729] T 385ef5d71f3a49c8b5057525a388230d P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ff3db459444683972a22263d29009b, State: Initialized, Role: FOLLOWER
I20260926 07:36:11.901853 13754 tablet_replica.cc:330] T 758385269fe544bfa148f596a1649143 P a6ff3db459444683972a22263d29009b: stopping tablet replica
I20260926 07:36:11.902535 13754 raft_consensus.cc:2229] T 758385269fe544bfa148f596a1649143 P a6ff3db459444683972a22263d29009b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:11.903024 13754 raft_consensus.cc:2258] T 758385269fe544bfa148f596a1649143 P a6ff3db459444683972a22263d29009b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:11.904394 13759 consensus_queue.cc:260] T 385ef5d71f3a49c8b5057525a388230d P a6ff3db459444683972a22263d29009b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:11.905267 13752 ts_tablet_manager.cc:1918] T 385ef5d71f3a49c8b5057525a388230d P 095560781b2d4e58b3804fce75312b90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:11.905725 13752 log.cc:1196] T 385ef5d71f3a49c8b5057525a388230d P 095560781b2d4e58b3804fce75312b90: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/385ef5d71f3a49c8b5057525a388230d
I20260926 07:36:11.906507 13752 ts_tablet_manager.cc:1939] T 385ef5d71f3a49c8b5057525a388230d P 095560781b2d4e58b3804fce75312b90: Deleting consensus metadata
I20260926 07:36:11.909233 13753 tablet_bootstrap.cc:654] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:11.912647 13754 ts_tablet_manager.cc:1905] T 758385269fe544bfa148f596a1649143 P a6ff3db459444683972a22263d29009b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:11.918829 13751 ts_tablet_manager.cc:1918] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:36:11.919322 13751 log.cc:1196] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/758385269fe544bfa148f596a1649143
I20260926 07:36:11.920742 13751 ts_tablet_manager.cc:1939] T 758385269fe544bfa148f596a1649143 P 16209ececd2d420588b390f95db9b92a: Deleting consensus metadata
I20260926 07:36:11.933612 13459 heartbeater.cc:503] Master 127.10.171.61:45551 was elected leader, sending a full tablet report...
I20260926 07:36:11.936681 13759 ts_tablet_manager.cc:1428] T 385ef5d71f3a49c8b5057525a388230d P a6ff3db459444683972a22263d29009b: Time spent starting tablet: real 0.059s user 0.010s sys 0.000s
I20260926 07:36:11.937628 13759 tablet_bootstrap.cc:492] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b: Bootstrap starting.
I20260926 07:36:11.941874 13754 ts_tablet_manager.cc:1918] T 758385269fe544bfa148f596a1649143 P a6ff3db459444683972a22263d29009b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:36:11.942224 13753 tablet_bootstrap.cc:492] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a: No bootstrap required, opened a new log
I20260926 07:36:11.942752 13753 ts_tablet_manager.cc:1397] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a: Time spent bootstrapping tablet: real 0.057s user 0.010s sys 0.000s
I20260926 07:36:11.944725 13754 log.cc:1196] T 758385269fe544bfa148f596a1649143 P a6ff3db459444683972a22263d29009b: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/758385269fe544bfa148f596a1649143
I20260926 07:36:11.943223 13759 tablet_bootstrap.cc:654] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:11.947217 13753 raft_consensus.cc:348] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:11.949488 13753 raft_consensus.cc:374] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:11.949777 13753 raft_consensus.cc:729] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16209ececd2d420588b390f95db9b92a, State: Initialized, Role: FOLLOWER
I20260926 07:36:11.946214 13754 ts_tablet_manager.cc:1939] T 758385269fe544bfa148f596a1649143 P a6ff3db459444683972a22263d29009b: Deleting consensus metadata
I20260926 07:36:11.952667 13754 tablet_replica.cc:330] T 385ef5d71f3a49c8b5057525a388230d P a6ff3db459444683972a22263d29009b: stopping tablet replica
I20260926 07:36:11.953280 13754 raft_consensus.cc:2229] T 385ef5d71f3a49c8b5057525a388230d P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:11.953717 13754 raft_consensus.cc:2258] T 385ef5d71f3a49c8b5057525a388230d P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:11.955439 13759 tablet_bootstrap.cc:492] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b: No bootstrap required, opened a new log
I20260926 07:36:11.955829 13759 ts_tablet_manager.cc:1397] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.006s
I20260926 07:36:11.958071 13759 raft_consensus.cc:348] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:11.958725 13759 raft_consensus.cc:374] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:11.958992 13759 raft_consensus.cc:729] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ff3db459444683972a22263d29009b, State: Initialized, Role: FOLLOWER
I20260926 07:36:11.959583 13759 consensus_queue.cc:260] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:11.960681 13753 consensus_queue.cc:260] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:11.967306 13753 ts_tablet_manager.cc:1428] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
I20260926 07:36:11.973882 13754 ts_tablet_manager.cc:1905] T 385ef5d71f3a49c8b5057525a388230d P a6ff3db459444683972a22263d29009b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:11.978223 13759 ts_tablet_manager.cc:1428] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
I20260926 07:36:11.989164 13754 ts_tablet_manager.cc:1918] T 385ef5d71f3a49c8b5057525a388230d P a6ff3db459444683972a22263d29009b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:11.989500 13754 log.cc:1196] T 385ef5d71f3a49c8b5057525a388230d P a6ff3db459444683972a22263d29009b: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/385ef5d71f3a49c8b5057525a388230d
I20260926 07:36:11.990159 13754 ts_tablet_manager.cc:1939] T 385ef5d71f3a49c8b5057525a388230d P a6ff3db459444683972a22263d29009b: Deleting consensus metadata
W20260926 07:36:12.125891 13349 connection.cc:535] server connection from 127.0.0.1:42394 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:36:12.125856 13193 connection.cc:535] server connection from 127.0.0.1:51478 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:36:12.126475 13193 connection.cc:535] client connection to 127.10.171.61:45551 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:36:12.127132 13347 connection.cc:535] client connection to 127.10.171.61:45551 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:36:12.127929 13036 connection.cc:535] server connection from 127.0.0.1:51078 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:36:12.128961 13643 connection.cc:535] server connection from 127.0.0.1:51950 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:36:12.131069 13471 connection.cc:535] client connection to 127.10.171.61:45551 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:36:12.132915 13471 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.10.171.61:45551)
W20260926 07:36:12.133663 13471 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.171.61:45551)
W20260926 07:36:12.134438 13471 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.10.171.61:45551)
W20260926 07:36:12.139051 13471 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.10.171.61:45551)
I20260926 07:36:12.153833 13770 raft_consensus.cc:482] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:12.154299 13770 raft_consensus.cc:504] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:12.156035 13770 leader_election.cc:288] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
I20260926 07:36:12.157521 13264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a83a4b4cbb4490b44acc66e4676c18" candidate_uuid: "a6ff3db459444683972a22263d29009b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16209ececd2d420588b390f95db9b92a" is_pre_election: true
I20260926 07:36:12.157582 13108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a83a4b4cbb4490b44acc66e4676c18" candidate_uuid: "a6ff3db459444683972a22263d29009b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "095560781b2d4e58b3804fce75312b90" is_pre_election: true
I20260926 07:36:12.158049 13264 raft_consensus.cc:2454] T b7a83a4b4cbb4490b44acc66e4676c18 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ff3db459444683972a22263d29009b in term 0.
I20260926 07:36:12.158252 13108 raft_consensus.cc:2454] T b7a83a4b4cbb4490b44acc66e4676c18 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ff3db459444683972a22263d29009b in term 0.
I20260926 07:36:12.159241 13348 leader_election.cc:302] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16209ececd2d420588b390f95db9b92a, a6ff3db459444683972a22263d29009b; no voters:
I20260926 07:36:12.159910 13770 raft_consensus.cc:2781] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:36:12.160336 13770 raft_consensus.cc:482] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:36:12.160625 13770 raft_consensus.cc:3037] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:36:12.166798 13770 raft_consensus.cc:504] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:12.169190 13108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a83a4b4cbb4490b44acc66e4676c18" candidate_uuid: "a6ff3db459444683972a22263d29009b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "095560781b2d4e58b3804fce75312b90"
I20260926 07:36:12.169692 13108 raft_consensus.cc:3037] T b7a83a4b4cbb4490b44acc66e4676c18 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:36:12.169764 13770 leader_election.cc:288] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 election: Requested vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
I20260926 07:36:12.170876 13264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a83a4b4cbb4490b44acc66e4676c18" candidate_uuid: "a6ff3db459444683972a22263d29009b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16209ececd2d420588b390f95db9b92a"
I20260926 07:36:12.171326 13264 raft_consensus.cc:3037] T b7a83a4b4cbb4490b44acc66e4676c18 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:36:12.175966 13264 raft_consensus.cc:2454] T b7a83a4b4cbb4490b44acc66e4676c18 P 16209ececd2d420588b390f95db9b92a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ff3db459444683972a22263d29009b in term 1.
I20260926 07:36:12.176084 13108 raft_consensus.cc:2454] T b7a83a4b4cbb4490b44acc66e4676c18 P 095560781b2d4e58b3804fce75312b90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ff3db459444683972a22263d29009b in term 1.
I20260926 07:36:12.176895 13348 leader_election.cc:302] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16209ececd2d420588b390f95db9b92a, a6ff3db459444683972a22263d29009b; no voters:
I20260926 07:36:12.177632 13770 raft_consensus.cc:2781] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:36:12.178671 13770 raft_consensus.cc:686] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [term 1 LEADER]: Becoming Leader. State: Replica: a6ff3db459444683972a22263d29009b, State: Running, Role: LEADER
I20260926 07:36:12.179558 13770 consensus_queue.cc:237] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:12.350366 13777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7726b3d3f73544318efe4a3be7a63f70)
I20260926 07:36:12.350782 13777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:12.353338 13777 leader_election.cc:288] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551)
W20260926 07:36:12.355827 13645 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:36:12.360195 13643 leader_election.cc:334] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551): Network error: Client connection negotiation failed: client connection to 127.10.171.61:45551: connect: Connection refused (error 111)
W20260926 07:36:12.360918 13645 leader_election.cc:334] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:36:12.361303 13645 leader_election.cc:302] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a6ef379bc314e798568d59eba3a3a91; no voters: 51342b641d47457d84f48fcbb6c48375, 7726b3d3f73544318efe4a3be7a63f70
I20260926 07:36:12.362088 13777 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260926 07:36:12.796226 13783 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:12.796664 13783 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:12.798359 13783 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:36:12.800279 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:36:12.800837 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:36:12.801210 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:36:12.802126 13783 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:36:13.105695 13785 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7726b3d3f73544318efe4a3be7a63f70)
I20260926 07:36:13.106092 13785 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:13.107738 13785 leader_election.cc:288] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551)
W20260926 07:36:13.113353 13645 leader_election.cc:334] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:36:13.113898 13643 leader_election.cc:334] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551): Network error: Client connection negotiation failed: client connection to 127.10.171.61:45551: connect: Connection refused (error 111)
I20260926 07:36:13.114313 13643 leader_election.cc:302] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a6ef379bc314e798568d59eba3a3a91; no voters: 51342b641d47457d84f48fcbb6c48375, 7726b3d3f73544318efe4a3be7a63f70
I20260926 07:36:13.114987 13785 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260926 07:36:13.146992 13789 consensus_queue.cc:1035] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [LEADER]: Connected to new peer: Peer: permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:36:13.182498 13792 consensus_queue.cc:1035] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [LEADER]: Connected to new peer: Peer: permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:36:13.834401 10924 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
/tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.171.61:45551
--webserver_interface=127.10.171.61
--webserver_port=44225
--builtin_ntp_servers=127.10.171.20:42033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615
--hive_metastore_uris=thrift://127.0.0.1:35621
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 07:36:14.042703 13802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7726b3d3f73544318efe4a3be7a63f70)
I20260926 07:36:14.043164 13802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:14.044827 13802 leader_election.cc:288] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551)
W20260926 07:36:14.050889 13645 leader_election.cc:334] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:36:14.051371 13643 leader_election.cc:334] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551): Network error: Client connection negotiation failed: client connection to 127.10.171.61:45551: connect: Connection refused (error 111)
I20260926 07:36:14.051746 13643 leader_election.cc:302] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a6ef379bc314e798568d59eba3a3a91; no voters: 51342b641d47457d84f48fcbb6c48375, 7726b3d3f73544318efe4a3be7a63f70
I20260926 07:36:14.052500 13802 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260926 07:36:14.153478 13801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 07:36:14.154060 13801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 07:36:14.154538 13801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 07:36:14.191617 13801 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260926 07:36:14.191924 13801 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260926 07:36:14.192271 13801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 07:36:14.192557 13801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 07:36:14.192828 13801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 07:36:14.193081 13801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 07:36:14.236773 13801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.171.20:42033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:35621
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.171.61:45551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.171.61
--webserver_port=44225
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 06:55:20 UTC on ccycloud.quasar-qwhbmo.root.comops.site
build id 4673737
TSAN enabled
I20260926 07:36:14.238268 13801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 07:36:14.239996 13801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 07:36:14.258919 13815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:36:15.273284 13828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7726b3d3f73544318efe4a3be7a63f70)
I20260926 07:36:15.273710 13828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:15.275259 13828 leader_election.cc:288] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551)
W20260926 07:36:15.281003 13643 leader_election.cc:334] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551): Network error: Client connection negotiation failed: client connection to 127.10.171.61:45551: connect: Connection refused (error 111)
W20260926 07:36:15.281435 13645 leader_election.cc:334] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:36:15.281904 13645 leader_election.cc:302] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a6ef379bc314e798568d59eba3a3a91; no voters: 51342b641d47457d84f48fcbb6c48375, 7726b3d3f73544318efe4a3be7a63f70
I20260926 07:36:15.282548 13828 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260926 07:36:14.259995 13818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:36:14.261296 13801 server_base.cc:1034] running on GCE node
W20260926 07:36:14.258962 13816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 07:36:15.514076 13801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 07:36:15.517177 13801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 07:36:15.518520 13801 hybrid_clock.cc:648] HybridClock initialized: now 1790408175518492 us; error 50 us; skew 500 ppm
I20260926 07:36:15.519351 13801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 07:36:15.530876 13801 webserver.cc:460] Webserver started at http://127.10.171.61:44225/ using document root <none> and password file <none>
I20260926 07:36:15.531951 13801 fs_manager.cc:359] Metadata directory not provided
I20260926 07:36:15.532218 13801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 07:36:15.541007 13801 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.005s
I20260926 07:36:15.545724 13836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 07:36:15.546798 13801 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260926 07:36:15.547168 13801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/data,/tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/wal
uuid: "7726b3d3f73544318efe4a3be7a63f70"
format_stamp: "Formatted at 2026-09-26 07:35:56 on dist-test-slave-3mmx"
I20260926 07:36:15.549679 13801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 07:36:15.593189 13801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 07:36:15.594623 13801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 07:36:15.595105 13801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 07:36:15.667589 13801 rpc_server.cc:307] RPC server started. Bound to: 127.10.171.61:45551
I20260926 07:36:15.667698 13887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.171.61:45551 every 8 connection(s)
I20260926 07:36:15.670608 13801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/master-1/data/info.pb
I20260926 07:36:15.677997 10924 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNvrOpy/build/tsan/bin/kudu as pid 13801
I20260926 07:36:15.681780 13888 sys_catalog.cc:263] Verifying existing consensus state
I20260926 07:36:15.687853 13888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70: Bootstrap starting.
I20260926 07:36:15.707033 13309 heartbeater.cc:347] Connected to a master server at 127.10.171.61:45551
I20260926 07:36:15.707620 13309 heartbeater.cc:503] Master 127.10.171.61:45551 was elected leader, sending a full tablet report...
I20260926 07:36:15.724994 13888 log.cc:824] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70: Log is configured to *not* fsync() on all Append() calls
I20260926 07:36:15.725935 13153 heartbeater.cc:347] Connected to a master server at 127.10.171.61:45551
I20260926 07:36:15.726656 13153 heartbeater.cc:503] Master 127.10.171.61:45551 was elected leader, sending a full tablet report...
I20260926 07:36:15.743415 13459 heartbeater.cc:347] Connected to a master server at 127.10.171.61:45551
I20260926 07:36:15.743968 13459 heartbeater.cc:503] Master 127.10.171.61:45551 was elected leader, sending a full tablet report...
I20260926 07:36:15.816303 13888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70: Bootstrap replayed 1/1 log segments. Stats: ops{read=35 overwritten=0 applied=35 ignored=0} inserts{seen=19 ignored=0} mutations{seen=29 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 07:36:15.817093 13888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70: Bootstrap complete.
I20260926 07:36:15.834441 13888 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:15.836615 13888 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7726b3d3f73544318efe4a3be7a63f70, State: Initialized, Role: FOLLOWER
I20260926 07:36:15.837357 13888 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 35, Last appended: 5.35, Last appended by leader: 35, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:15.840037 13896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:15.840740 13896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:36:15.841254 13888 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 07:36:15.885762 13911 catalog_manager.cc:1259] Loaded cluster ID: 6e2e15d6975946d1a4009a3693fa16a8
I20260926 07:36:15.886052 13911 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70: loading cluster ID for follower catalog manager: success
I20260926 07:36:15.895393 13911 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70: acquiring CA information for follower catalog manager: success
I20260926 07:36:15.902330 13911 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 07:36:15.870347 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:36:15.906697 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:36:15.926406 13530 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:36:15.926867 13530 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:36:16.101210 13915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 7726b3d3f73544318efe4a3be7a63f70)
I20260926 07:36:16.101632 13915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:16.103221 13915 leader_election.cc:288] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551)
W20260926 07:36:16.108400 13645 leader_election.cc:334] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:36:16.115499 13863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a6ef379bc314e798568d59eba3a3a91" candidate_term: 6 candidate_status { last_received { term: 5 index: 35 } } ignore_live_leader: false dest_uuid: "7726b3d3f73544318efe4a3be7a63f70" is_pre_election: true
I20260926 07:36:16.116340 13863 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a6ef379bc314e798568d59eba3a3a91 in term 5.
I20260926 07:36:16.117401 13643 leader_election.cc:302] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a6ef379bc314e798568d59eba3a3a91, 7726b3d3f73544318efe4a3be7a63f70; no voters: 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:16.118559 13915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260926 07:36:16.118950 13915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 7726b3d3f73544318efe4a3be7a63f70)
I20260926 07:36:16.119333 13915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 5 FOLLOWER]: Advancing to term 6
I20260926 07:36:16.124699 13915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:16.126255 13915 leader_election.cc:288] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 election: Requested vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 7726b3d3f73544318efe4a3be7a63f70 (127.10.171.61:45551)
I20260926 07:36:16.127787 13863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a6ef379bc314e798568d59eba3a3a91" candidate_term: 6 candidate_status { last_received { term: 5 index: 35 } } ignore_live_leader: false dest_uuid: "7726b3d3f73544318efe4a3be7a63f70"
I20260926 07:36:16.128381 13863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 5 FOLLOWER]: Advancing to term 6
W20260926 07:36:16.129498 13645 leader_election.cc:334] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:36:16.135336 13863 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a6ef379bc314e798568d59eba3a3a91 in term 6.
I20260926 07:36:16.138535 13643 leader_election.cc:302] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a6ef379bc314e798568d59eba3a3a91, 7726b3d3f73544318efe4a3be7a63f70; no voters: 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:16.139312 13915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 6 FOLLOWER]: Leader election won for term 6
I20260926 07:36:16.140674 13915 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [term 6 LEADER]: Becoming Leader. State: Replica: 0a6ef379bc314e798568d59eba3a3a91, State: Running, Role: LEADER
I20260926 07:36:16.141455 13915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 35, Committed index: 35, Last appended: 5.35, Last appended by leader: 35, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:16.145607 13918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a6ef379bc314e798568d59eba3a3a91. Latest consensus state: current_term: 6 leader_uuid: "0a6ef379bc314e798568d59eba3a3a91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:16.146353 13918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: This master's current role is: LEADER
I20260926 07:36:16.148403 13920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 07:36:16.157236 13920 catalog_manager.cc:669] Loaded metadata for table default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]
I20260926 07:36:16.158913 13920 catalog_manager.cc:669] Loaded metadata for table default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]
I20260926 07:36:16.160465 13920 catalog_manager.cc:669] Loaded metadata for table default.table_5280e066b7404013a7c7bdfe8e9bc18b [id=e093563ca611494caed51759972753c6]
I20260926 07:36:16.167747 13920 tablet_loader.cc:96] loaded metadata for tablet 23b91b8716e5405e8b3aad7df68183fc (table default.table_5280e066b7404013a7c7bdfe8e9bc18b [id=e093563ca611494caed51759972753c6])
I20260926 07:36:16.170564 13920 tablet_loader.cc:96] loaded metadata for tablet 3efbc3ccb32d46919864ad44f7cf630d (table default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6])
I20260926 07:36:16.172617 13920 tablet_loader.cc:96] loaded metadata for tablet b7a83a4b4cbb4490b44acc66e4676c18 (table default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f])
I20260926 07:36:16.173972 13920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 07:36:16.180159 13920 catalog_manager.cc:1259] Loaded cluster ID: 6e2e15d6975946d1a4009a3693fa16a8
I20260926 07:36:16.180402 13920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 07:36:16.186636 13920 catalog_manager.cc:1504] Loading token signing keys...
I20260926 07:36:16.192142 13920 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: Loaded TSK: 0
I20260926 07:36:16.193683 13920 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 07:36:16.198668 13920 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260926 07:36:16.203612 13920 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260926 07:36:16.213065 13309 heartbeater.cc:503] Master 127.10.171.61:45551 was elected leader, sending a full tablet report...
I20260926 07:36:16.225099 13852 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "16209ececd2d420588b390f95db9b92a" instance_seqno: 1790408164464241) as {username='slave'} at 127.10.171.2:41071; Asking this server to re-register.
I20260926 07:36:16.226694 13309 heartbeater.cc:464] Registering TS with master...
I20260926 07:36:16.230350 13852 ts_manager.cc:194] Registered new tserver with Master: 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
I20260926 07:36:16.231159 13153 heartbeater.cc:503] Master 127.10.171.61:45551 was elected leader, sending a full tablet report...
I20260926 07:36:16.235116 13852 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "095560781b2d4e58b3804fce75312b90" instance_seqno: 1790408162365667) as {username='slave'} at 127.10.171.1:47627; Asking this server to re-register.
I20260926 07:36:16.236471 13153 heartbeater.cc:464] Registering TS with master...
I20260926 07:36:16.237340 13656 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51938:
name: "default.Table_6d097d04d5e6412d9f5354f5c40d580f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:36:16.238585 13863 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Refusing update from remote peer 0a6ef379bc314e798568d59eba3a3a91: Log matching property violated. Preceding OpId in replica: term: 5 index: 35. Preceding OpId from leader: term: 6 index: 37. (index mismatch)
I20260926 07:36:16.239068 13655 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51938:
table { table_name: "default.Table_708d07ef404b42629d7c958b81980ca1" } new_table_name: "default.Table_9fbcc2f37250434e80c28330fb80f80b" modify_external_catalogs: true
I20260926 07:36:16.240964 13653 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51938:
table { table_name: "default.Table_5280e066b7404013a7c7bdfe8e9bc18b" } new_table_name: "default.Table_b2944ae24c6f47d4a1c3cb780994e4fe" modify_external_catalogs: true
I20260926 07:36:16.243424 13652 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51938:
table { table_name: "default.Table_b9246fcaf5ed4103b8c7835af689efab" } modify_external_catalogs: true
I20260926 07:36:16.244160 13652 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51938:
table { table_name: "default.Table_b9246fcaf5ed4103b8c7835af689efab" } modify_external_catalogs: true
W20260926 07:36:16.248575 13645 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 -> Peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Couldn't send request to peer 51342b641d47457d84f48fcbb6c48375. Status: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 07:36:16.248708 13654 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51938:
name: "default.Table_daf8f6d0f2c6493d879b9a3bfbd1cfad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:36:16.250051 13657 master_service.cc:946] ReplaceTablet: received request to replace tablet b7a83a4b4cbb4490b44acc66e4676c18 from {username='slave'} at 127.0.0.1:51938
I20260926 07:36:16.250180 13651 master_service.cc:946] ReplaceTablet: received request to replace tablet b7a83a4b4cbb4490b44acc66e4676c18 from {username='slave'} at 127.0.0.1:51938
I20260926 07:36:16.254254 13852 ts_manager.cc:194] Registered new tserver with Master: 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253)
I20260926 07:36:16.251649 13915 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 36, Last known committed idx: 35, Time since last communication: 0.001s
F20260926 07:36:16.255998 13715 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_708d07ef404b42629d7c958b81980ca1
I20260926 07:36:16.257000 13459 heartbeater.cc:503] Master 127.10.171.61:45551 was elected leader, sending a full tablet report...
*** Check failure stack trace: ***
I20260926 07:36:16.252821 13650 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51938:
name: "default.Table_5af6599ea22546fdb4c598d6a124f28d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:36:16.256558 13658 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51938:
name: "default.Table_8019d82e72b54af899515d80cb8aa33e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 07:36:16.269666 13896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a6ef379bc314e798568d59eba3a3a91. Latest consensus state: current_term: 6 leader_uuid: "0a6ef379bc314e798568d59eba3a3a91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:16.270390 13896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:36:16.274045 13852 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "a6ff3db459444683972a22263d29009b" instance_seqno: 1790408166437023) as {username='slave'} at 127.10.171.3:35501; Asking this server to re-register.
I20260926 07:36:16.275511 13459 heartbeater.cc:464] Registering TS with master...
I20260926 07:36:16.279834 13923 mvcc.cc:205] Tried to move back new op lower bound from 7333511889814855680 to 7333511889482784768. Current Snapshot: MvccSnapshot[applied={T|T < 7333511889814855680}]
@ 0x7fac1309d9a0 google::LogMessage::Flush() at ??:0
I20260926 07:36:16.285892 13852 ts_manager.cc:194] Registered new tserver with Master: a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
I20260926 07:36:16.287154 13896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "0a6ef379bc314e798568d59eba3a3a91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:16.287822 13896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 07:36:16.291847 13922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "0a6ef379bc314e798568d59eba3a3a91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } } }
I20260926 07:36:16.292676 13922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91 [sys.catalog]: This master's current role is: LEADER
@ 0x7fac1309d70f google::LogMessage::~LogMessage() at ??:0
I20260926 07:36:16.333743 13651 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: Sending DeleteTablet for 3 replicas of tablet b7a83a4b4cbb4490b44acc66e4676c18
I20260926 07:36:16.335526 13651 catalog_manager.cc:6565] ReplaceTablet: tablet b7a83a4b4cbb4490b44acc66e4676c18 deleted and replaced by tablet 504c993884da427fb5dd210b73dd41af
I20260926 07:36:16.351897 13087 tablet_service.cc:1467] Processing CreateTablet for tablet 23b91b8716e5405e8b3aad7df68183fc (DEFAULT_TABLE table=default.table_5280e066b7404013a7c7bdfe8e9bc18b [id=e093563ca611494caed51759972753c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:16.353189 13087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23b91b8716e5405e8b3aad7df68183fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:16.357306 13086 tablet_service.cc:1514] Processing DeleteTablet for tablet b7a83a4b4cbb4490b44acc66e4676c18 with delete_type TABLET_DATA_DELETED (replaced by tablet 504c993884da427fb5dd210b73dd41af) from {username='slave'} at 127.0.0.1:51090
I20260926 07:36:16.357689 13088 tablet_service.cc:1467] Processing CreateTablet for tablet 3efbc3ccb32d46919864ad44f7cf630d (DEFAULT_TABLE table=default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:16.358712 13088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3efbc3ccb32d46919864ad44f7cf630d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:16.359368 13934 tablet_replica.cc:330] T b7a83a4b4cbb4490b44acc66e4676c18 P 095560781b2d4e58b3804fce75312b90: stopping tablet replica
I20260926 07:36:16.360033 13934 raft_consensus.cc:2229] T b7a83a4b4cbb4490b44acc66e4676c18 P 095560781b2d4e58b3804fce75312b90 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:16.361366 13934 raft_consensus.cc:2258] T b7a83a4b4cbb4490b44acc66e4676c18 P 095560781b2d4e58b3804fce75312b90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:16.361970 13244 tablet_service.cc:1467] Processing CreateTablet for tablet 23b91b8716e5405e8b3aad7df68183fc (DEFAULT_TABLE table=default.table_5280e066b7404013a7c7bdfe8e9bc18b [id=e093563ca611494caed51759972753c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:16.363123 13244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23b91b8716e5405e8b3aad7df68183fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:16.363266 13934 ts_tablet_manager.cc:1905] T b7a83a4b4cbb4490b44acc66e4676c18 P 095560781b2d4e58b3804fce75312b90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:16.367182 13242 tablet_service.cc:1467] Processing CreateTablet for tablet 3efbc3ccb32d46919864ad44f7cf630d (DEFAULT_TABLE table=default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:16.368242 13242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3efbc3ccb32d46919864ad44f7cf630d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:16.370998 13243 tablet_service.cc:1514] Processing DeleteTablet for tablet b7a83a4b4cbb4490b44acc66e4676c18 with delete_type TABLET_DATA_DELETED (replaced by tablet 504c993884da427fb5dd210b73dd41af) from {username='slave'} at 127.0.0.1:51490
I20260926 07:36:16.383677 13934 ts_tablet_manager.cc:1918] T b7a83a4b4cbb4490b44acc66e4676c18 P 095560781b2d4e58b3804fce75312b90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:36:16.384032 13934 log.cc:1196] T b7a83a4b4cbb4490b44acc66e4676c18 P 095560781b2d4e58b3804fce75312b90: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/b7a83a4b4cbb4490b44acc66e4676c18
I20260926 07:36:16.385941 13934 ts_tablet_manager.cc:1939] T b7a83a4b4cbb4490b44acc66e4676c18 P 095560781b2d4e58b3804fce75312b90: Deleting consensus metadata
I20260926 07:36:16.396811 13935 tablet_replica.cc:330] T b7a83a4b4cbb4490b44acc66e4676c18 P 16209ececd2d420588b390f95db9b92a: stopping tablet replica
I20260926 07:36:16.397614 13935 raft_consensus.cc:2229] T b7a83a4b4cbb4490b44acc66e4676c18 P 16209ececd2d420588b390f95db9b92a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:16.398269 13935 raft_consensus.cc:2258] T b7a83a4b4cbb4490b44acc66e4676c18 P 16209ececd2d420588b390f95db9b92a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:16.400658 13643 catalog_manager.cc:4841] TS 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): tablet b7a83a4b4cbb4490b44acc66e4676c18 (table default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]) successfully deleted
I20260926 07:36:16.403647 13936 tablet_bootstrap.cc:492] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90: Bootstrap starting.
I20260926 07:36:16.400419 13935 ts_tablet_manager.cc:1905] T b7a83a4b4cbb4490b44acc66e4676c18 P 16209ececd2d420588b390f95db9b92a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 07:36:16.416420 13347 consensus_peers.cc:487] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7a83a4b4cbb4490b44acc66e4676c18. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:36:16.418921 13937 tablet_bootstrap.cc:492] T 23b91b8716e5405e8b3aad7df68183fc P 16209ececd2d420588b390f95db9b92a: Bootstrap starting.
I20260926 07:36:16.421715 13936 tablet_bootstrap.cc:654] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:16.424811 13242 tablet_service.cc:1467] Processing CreateTablet for tablet 504c993884da427fb5dd210b73dd41af (DEFAULT_TABLE table=default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:16.425967 13242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 504c993884da427fb5dd210b73dd41af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:16.426944 13937 tablet_bootstrap.cc:654] T 23b91b8716e5405e8b3aad7df68183fc P 16209ececd2d420588b390f95db9b92a: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:16.427837 13087 tablet_service.cc:1467] Processing CreateTablet for tablet 504c993884da427fb5dd210b73dd41af (DEFAULT_TABLE table=default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:16.428979 13087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 504c993884da427fb5dd210b73dd41af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:16.439848 13394 tablet_service.cc:1467] Processing CreateTablet for tablet 23b91b8716e5405e8b3aad7df68183fc (DEFAULT_TABLE table=default.table_5280e066b7404013a7c7bdfe8e9bc18b [id=e093563ca611494caed51759972753c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:16.441304 13394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23b91b8716e5405e8b3aad7df68183fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:16.444970 13392 tablet_service.cc:1467] Processing CreateTablet for tablet 3efbc3ccb32d46919864ad44f7cf630d (DEFAULT_TABLE table=default.table_b9246fcaf5ed4103b8c7835af689efab [id=5eab828232e7481fa0f830a29a32f1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:16.446007 13392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3efbc3ccb32d46919864ad44f7cf630d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:16.448513 13393 tablet_service.cc:1514] Processing DeleteTablet for tablet b7a83a4b4cbb4490b44acc66e4676c18 with delete_type TABLET_DATA_DELETED (replaced by tablet 504c993884da427fb5dd210b73dd41af) from {username='slave'} at 127.0.0.1:42408
I20260926 07:36:16.450124 13391 tablet_service.cc:1467] Processing CreateTablet for tablet 504c993884da427fb5dd210b73dd41af (DEFAULT_TABLE table=default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:16.451185 13391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 504c993884da427fb5dd210b73dd41af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:16.451465 13936 tablet_bootstrap.cc:492] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90: No bootstrap required, opened a new log
I20260926 07:36:16.451941 13936 ts_tablet_manager.cc:1397] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90: Time spent bootstrapping tablet: real 0.048s user 0.011s sys 0.000s
I20260926 07:36:16.458702 13935 ts_tablet_manager.cc:1918] T b7a83a4b4cbb4490b44acc66e4676c18 P 16209ececd2d420588b390f95db9b92a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:36:16.459151 13935 log.cc:1196] T b7a83a4b4cbb4490b44acc66e4676c18 P 16209ececd2d420588b390f95db9b92a: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/b7a83a4b4cbb4490b44acc66e4676c18
@ 0x7fac1e138aea kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
W20260926 07:36:16.464048 13348 consensus_peers.cc:487] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b -> Peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Couldn't send request to peer 16209ececd2d420588b390f95db9b92a. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:36:16.455372 13936 raft_consensus.cc:348] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.465438 13936 raft_consensus.cc:374] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:16.465713 13936 raft_consensus.cc:729] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 095560781b2d4e58b3804fce75312b90, State: Initialized, Role: FOLLOWER
I20260926 07:36:16.466709 13935 ts_tablet_manager.cc:1939] T b7a83a4b4cbb4490b44acc66e4676c18 P 16209ececd2d420588b390f95db9b92a: Deleting consensus metadata
I20260926 07:36:16.469825 13937 tablet_bootstrap.cc:492] T 23b91b8716e5405e8b3aad7df68183fc P 16209ececd2d420588b390f95db9b92a: No bootstrap required, opened a new log
I20260926 07:36:16.469128 13936 consensus_queue.cc:260] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.470222 13937 ts_tablet_manager.cc:1397] T 23b91b8716e5405e8b3aad7df68183fc P 16209ececd2d420588b390f95db9b92a: Time spent bootstrapping tablet: real 0.052s user 0.008s sys 0.017s
I20260926 07:36:16.471735 13644 catalog_manager.cc:4841] TS 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): tablet b7a83a4b4cbb4490b44acc66e4676c18 (table default.table_708d07ef404b42629d7c958b81980ca1 [id=762c449a1c174102af79fc5e64f6502f]) successfully deleted
I20260926 07:36:16.480706 13937 raft_consensus.cc:348] T 23b91b8716e5405e8b3aad7df68183fc P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.481769 13937 raft_consensus.cc:374] T 23b91b8716e5405e8b3aad7df68183fc P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:16.482129 13937 raft_consensus.cc:729] T 23b91b8716e5405e8b3aad7df68183fc P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16209ececd2d420588b390f95db9b92a, State: Initialized, Role: FOLLOWER
I20260926 07:36:16.484066 13937 consensus_queue.cc:260] T 23b91b8716e5405e8b3aad7df68183fc P 16209ececd2d420588b390f95db9b92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
@ 0x7fac1e13833c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7fac1e13bfd2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260926 07:36:16.517463 13154 heartbeater.cc:503] Master 127.10.171.60:45615 was elected leader, sending a full tablet report...
I20260926 07:36:16.520422 13936 ts_tablet_manager.cc:1428] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90: Time spent starting tablet: real 0.068s user 0.024s sys 0.028s
I20260926 07:36:16.521307 13936 tablet_bootstrap.cc:492] T 23b91b8716e5405e8b3aad7df68183fc P 095560781b2d4e58b3804fce75312b90: Bootstrap starting.
I20260926 07:36:16.524369 13941 tablet_replica.cc:330] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b: stopping tablet replica
I20260926 07:36:16.525040 13941 raft_consensus.cc:2229] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [term 1 LEADER]: Raft consensus shutting down.
I20260926 07:36:16.526109 13941 raft_consensus.cc:2258] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:16.527101 13936 tablet_bootstrap.cc:654] T 23b91b8716e5405e8b3aad7df68183fc P 095560781b2d4e58b3804fce75312b90: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:16.543431 13651 master_service.cc:946] ReplaceTablet: received request to replace tablet 23b91b8716e5405e8b3aad7df68183fc from {username='slave'} at 127.0.0.1:51938
I20260926 07:36:16.544173 13657 master_service.cc:946] ReplaceTablet: received request to replace tablet 504c993884da427fb5dd210b73dd41af from {username='slave'} at 127.0.0.1:51938
@ 0x7fac1e13bf8a _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260926 07:36:16.550904 13936 tablet_bootstrap.cc:492] T 23b91b8716e5405e8b3aad7df68183fc P 095560781b2d4e58b3804fce75312b90: No bootstrap required, opened a new log
I20260926 07:36:16.551316 13936 ts_tablet_manager.cc:1397] T 23b91b8716e5405e8b3aad7df68183fc P 095560781b2d4e58b3804fce75312b90: Time spent bootstrapping tablet: real 0.030s user 0.016s sys 0.007s
I20260926 07:36:16.556211 13941 ts_tablet_manager.cc:1905] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:16.558367 13936 raft_consensus.cc:348] T 23b91b8716e5405e8b3aad7df68183fc P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.559049 13936 raft_consensus.cc:374] T 23b91b8716e5405e8b3aad7df68183fc P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:16.559314 13936 raft_consensus.cc:729] T 23b91b8716e5405e8b3aad7df68183fc P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 095560781b2d4e58b3804fce75312b90, State: Initialized, Role: FOLLOWER
I20260926 07:36:16.559962 13936 consensus_queue.cc:260] T 23b91b8716e5405e8b3aad7df68183fc P 095560781b2d4e58b3804fce75312b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.562000 13936 ts_tablet_manager.cc:1428] T 23b91b8716e5405e8b3aad7df68183fc P 095560781b2d4e58b3804fce75312b90: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260926 07:36:16.562824 13936 tablet_bootstrap.cc:492] T 504c993884da427fb5dd210b73dd41af P 095560781b2d4e58b3804fce75312b90: Bootstrap starting.
I20260926 07:36:16.579662 13937 ts_tablet_manager.cc:1428] T 23b91b8716e5405e8b3aad7df68183fc P 16209ececd2d420588b390f95db9b92a: Time spent starting tablet: real 0.109s user 0.035s sys 0.018s
I20260926 07:36:16.580988 13936 tablet_bootstrap.cc:654] T 504c993884da427fb5dd210b73dd41af P 095560781b2d4e58b3804fce75312b90: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:16.580282 13310 heartbeater.cc:503] Master 127.10.171.60:45615 was elected leader, sending a full tablet report...
I20260926 07:36:16.586541 13937 tablet_bootstrap.cc:492] T 3efbc3ccb32d46919864ad44f7cf630d P 16209ececd2d420588b390f95db9b92a: Bootstrap starting.
@ 0x7fac1e13bf1a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
I20260926 07:36:16.602056 13936 tablet_bootstrap.cc:492] T 504c993884da427fb5dd210b73dd41af P 095560781b2d4e58b3804fce75312b90: No bootstrap required, opened a new log
I20260926 07:36:16.602458 13936 ts_tablet_manager.cc:1397] T 504c993884da427fb5dd210b73dd41af P 095560781b2d4e58b3804fce75312b90: Time spent bootstrapping tablet: real 0.040s user 0.000s sys 0.012s
I20260926 07:36:16.602775 13937 tablet_bootstrap.cc:654] T 3efbc3ccb32d46919864ad44f7cf630d P 16209ececd2d420588b390f95db9b92a: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:16.604947 13936 raft_consensus.cc:348] T 504c993884da427fb5dd210b73dd41af P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.605835 13936 raft_consensus.cc:374] T 504c993884da427fb5dd210b73dd41af P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:16.606142 13936 raft_consensus.cc:729] T 504c993884da427fb5dd210b73dd41af P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 095560781b2d4e58b3804fce75312b90, State: Initialized, Role: FOLLOWER
I20260926 07:36:16.606961 13936 consensus_queue.cc:260] T 504c993884da427fb5dd210b73dd41af P 095560781b2d4e58b3804fce75312b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.608963 13651 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: Sending DeleteTablet for 3 replicas of tablet 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:36:16.610174 13936 ts_tablet_manager.cc:1428] T 504c993884da427fb5dd210b73dd41af P 095560781b2d4e58b3804fce75312b90: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260926 07:36:16.610297 13651 catalog_manager.cc:6565] ReplaceTablet: tablet 23b91b8716e5405e8b3aad7df68183fc deleted and replaced by tablet b310b46797af437ba5d7e5c11195b217
I20260926 07:36:16.611478 13087 tablet_service.cc:1514] Processing DeleteTablet for tablet 23b91b8716e5405e8b3aad7df68183fc with delete_type TABLET_DATA_DELETED (replaced by tablet b310b46797af437ba5d7e5c11195b217) from {username='slave'} at 127.0.0.1:51090
I20260926 07:36:16.613076 13934 tablet_replica.cc:330] T 23b91b8716e5405e8b3aad7df68183fc P 095560781b2d4e58b3804fce75312b90: stopping tablet replica
I20260926 07:36:16.613796 13934 raft_consensus.cc:2229] T 23b91b8716e5405e8b3aad7df68183fc P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:16.614214 13934 raft_consensus.cc:2258] T 23b91b8716e5405e8b3aad7df68183fc P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260926 07:36:16.615989 13651 catalog_manager.cc:5328] Ignoring report from unknown tablet 385ef5d71f3a49c8b5057525a388230d
I20260926 07:36:16.616611 13394 tablet_service.cc:1514] Processing DeleteTablet for tablet 23b91b8716e5405e8b3aad7df68183fc with delete_type TABLET_DATA_DELETED (replaced by tablet b310b46797af437ba5d7e5c11195b217) from {username='slave'} at 127.0.0.1:42408
I20260926 07:36:16.617277 13934 ts_tablet_manager.cc:1905] T 23b91b8716e5405e8b3aad7df68183fc P 095560781b2d4e58b3804fce75312b90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:16.622640 13937 tablet_bootstrap.cc:492] T 3efbc3ccb32d46919864ad44f7cf630d P 16209ececd2d420588b390f95db9b92a: No bootstrap required, opened a new log
I20260926 07:36:16.623072 13937 ts_tablet_manager.cc:1397] T 3efbc3ccb32d46919864ad44f7cf630d P 16209ececd2d420588b390f95db9b92a: Time spent bootstrapping tablet: real 0.037s user 0.011s sys 0.004s
@ 0x7fac1e13bee2 std::__1::__function::__alloc_func<>::operator()() at ??:0
I20260926 07:36:16.626003 13242 tablet_service.cc:1514] Processing DeleteTablet for tablet 23b91b8716e5405e8b3aad7df68183fc with delete_type TABLET_DATA_DELETED (replaced by tablet b310b46797af437ba5d7e5c11195b217) from {username='slave'} at 127.0.0.1:51490
I20260926 07:36:16.626768 13935 tablet_replica.cc:330] T 23b91b8716e5405e8b3aad7df68183fc P 16209ececd2d420588b390f95db9b92a: stopping tablet replica
I20260926 07:36:16.627414 13935 raft_consensus.cc:2229] T 23b91b8716e5405e8b3aad7df68183fc P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:16.627863 13935 raft_consensus.cc:2258] T 23b91b8716e5405e8b3aad7df68183fc P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:16.630224 13935 ts_tablet_manager.cc:1905] T 23b91b8716e5405e8b3aad7df68183fc P 16209ececd2d420588b390f95db9b92a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:16.632757 13943 raft_consensus.cc:482] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:16.633234 13943 raft_consensus.cc:504] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.635309 13943 leader_election.cc:288] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6ff3db459444683972a22263d29009b (127.10.171.3:34663), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
I20260926 07:36:16.637286 13413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3efbc3ccb32d46919864ad44f7cf630d" candidate_uuid: "095560781b2d4e58b3804fce75312b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ff3db459444683972a22263d29009b" is_pre_election: true
W20260926 07:36:16.639011 13034 leader_election.cc:341] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:36:16.641162 13264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3efbc3ccb32d46919864ad44f7cf630d" candidate_uuid: "095560781b2d4e58b3804fce75312b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16209ececd2d420588b390f95db9b92a" is_pre_election: true
I20260926 07:36:16.642839 13242 tablet_service.cc:1514] Processing DeleteTablet for tablet 504c993884da427fb5dd210b73dd41af with delete_type TABLET_DATA_DELETED (replaced by tablet 17525284c1734cf4ba71e1335654a4c5) from {username='slave'} at 127.0.0.1:51490
@ 0x7fac1e13b1de std::__1::__function::__func<>::operator()() at ??:0
@ 0x580785 std::__1::__function::__value_func<>::operator()() at ??:?
@ 0x5805ba std::__1::function<>::operator()() at ??:?
I20260926 07:36:16.652472 13087 tablet_service.cc:1514] Processing DeleteTablet for tablet 504c993884da427fb5dd210b73dd41af with delete_type TABLET_DATA_DELETED (replaced by tablet 17525284c1734cf4ba71e1335654a4c5) from {username='slave'} at 127.0.0.1:51090
I20260926 07:36:16.652849 13657 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a6ef379bc314e798568d59eba3a3a91: Sending DeleteTablet for 3 replicas of tablet 504c993884da427fb5dd210b73dd41af
I20260926 07:36:16.654350 13657 catalog_manager.cc:6565] ReplaceTablet: tablet 504c993884da427fb5dd210b73dd41af deleted and replaced by tablet 17525284c1734cf4ba71e1335654a4c5
I20260926 07:36:16.654762 13087 tablet_service.cc:1514] Processing DeleteTablet for tablet 504c993884da427fb5dd210b73dd41af with delete_type TABLET_DATA_DELETED (replaced by tablet 17525284c1734cf4ba71e1335654a4c5) from {username='slave'} at 127.0.0.1:51090
I20260926 07:36:16.656253 13394 tablet_service.cc:1514] Processing DeleteTablet for tablet 504c993884da427fb5dd210b73dd41af with delete_type TABLET_DATA_DELETED (replaced by tablet 17525284c1734cf4ba71e1335654a4c5) from {username='slave'} at 127.0.0.1:42408
I20260926 07:36:16.625523 13937 raft_consensus.cc:348] T 3efbc3ccb32d46919864ad44f7cf630d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.657356 13937 raft_consensus.cc:374] T 3efbc3ccb32d46919864ad44f7cf630d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:16.657554 13242 tablet_service.cc:1514] Processing DeleteTablet for tablet 504c993884da427fb5dd210b73dd41af with delete_type TABLET_DATA_DELETED (replaced by tablet 17525284c1734cf4ba71e1335654a4c5) from {username='slave'} at 127.0.0.1:51490
I20260926 07:36:16.657641 13937 raft_consensus.cc:729] T 3efbc3ccb32d46919864ad44f7cf630d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16209ececd2d420588b390f95db9b92a, State: Initialized, Role: FOLLOWER
I20260926 07:36:16.658299 13937 consensus_queue.cc:260] T 3efbc3ccb32d46919864ad44f7cf630d P 16209ececd2d420588b390f95db9b92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.659214 13264 raft_consensus.cc:2454] T 3efbc3ccb32d46919864ad44f7cf630d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 095560781b2d4e58b3804fce75312b90 in term 0.
I20260926 07:36:16.663928 13035 leader_election.cc:302] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a; no voters: a6ff3db459444683972a22263d29009b
I20260926 07:36:16.665740 13947 tablet_bootstrap.cc:492] T 3efbc3ccb32d46919864ad44f7cf630d P a6ff3db459444683972a22263d29009b: Bootstrap starting.
I20260926 07:36:16.668910 13937 ts_tablet_manager.cc:1428] T 3efbc3ccb32d46919864ad44f7cf630d P 16209ececd2d420588b390f95db9b92a: Time spent starting tablet: real 0.046s user 0.006s sys 0.001s
I20260926 07:36:16.669399 13943 raft_consensus.cc:2781] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:36:16.669772 13943 raft_consensus.cc:482] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:36:16.670130 13943 raft_consensus.cc:3037] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Advancing to term 1
@ 0x7fac1524edc7 kudu::Thread::SuperviseThread() at ??:0
@ 0x4230f0 __tsan_thread_start_func at ??:?
I20260926 07:36:16.680347 13944 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:16.680758 13944 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:16.682713 13944 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:36:16.685812 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:36:16.686436 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:36:16.686596 13937 tablet_bootstrap.cc:492] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a: Bootstrap starting.
I20260926 07:36:16.686792 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
@ 0x7fac168bc6db start_thread at ??:0
I20260926 07:36:16.687816 13944 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:36:16.696044 13242 tablet_service.cc:1467] Processing CreateTablet for tablet b310b46797af437ba5d7e5c11195b217 (DEFAULT_TABLE table=default.table_5280e066b7404013a7c7bdfe8e9bc18b [id=e093563ca611494caed51759972753c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:16.695941 13943 raft_consensus.cc:504] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.697168 13242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b310b46797af437ba5d7e5c11195b217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:16.697587 13087 tablet_service.cc:1467] Processing CreateTablet for tablet b310b46797af437ba5d7e5c11195b217 (DEFAULT_TABLE table=default.table_5280e066b7404013a7c7bdfe8e9bc18b [id=e093563ca611494caed51759972753c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 07:36:16.698559 13087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b310b46797af437ba5d7e5c11195b217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:16.698832 13413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3efbc3ccb32d46919864ad44f7cf630d" candidate_uuid: "095560781b2d4e58b3804fce75312b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ff3db459444683972a22263d29009b"
W20260926 07:36:16.700196 13034 leader_election.cc:341] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Illegal state: must be running to vote when last-logged opid is not known
I20260926 07:36:16.702046 13264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3efbc3ccb32d46919864ad44f7cf630d" candidate_uuid: "095560781b2d4e58b3804fce75312b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16209ececd2d420588b390f95db9b92a"
@ 0x7fac11d4261f clone at ??:0
I20260926 07:36:16.702608 13264 raft_consensus.cc:3037] T 3efbc3ccb32d46919864ad44f7cf630d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:36:16.704378 13943 leader_election.cc:288] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [CANDIDATE]: Term 1 election: Requested vote from peers a6ff3db459444683972a22263d29009b (127.10.171.3:34663), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
I20260926 07:36:16.705132 13947 tablet_bootstrap.cc:654] T 3efbc3ccb32d46919864ad44f7cf630d P a6ff3db459444683972a22263d29009b: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:16.707266 13941 ts_tablet_manager.cc:1918] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 07:36:16.707548 13941 log.cc:1196] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-2/wal/wals/b7a83a4b4cbb4490b44acc66e4676c18
I20260926 07:36:16.708294 13941 ts_tablet_manager.cc:1939] T b7a83a4b4cbb4490b44acc66e4676c18 P a6ff3db459444683972a22263d29009b: Deleting consensus metadata
I20260926 07:36:16.716619 13264 raft_consensus.cc:2454] T 3efbc3ccb32d46919864ad44f7cf630d P 16209ececd2d420588b390f95db9b92a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 095560781b2d4e58b3804fce75312b90 in term 1.
I20260926 07:36:16.717600 13035 leader_election.cc:302] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a; no voters: a6ff3db459444683972a22263d29009b
I20260926 07:36:16.718014 13935 ts_tablet_manager.cc:1918] T 23b91b8716e5405e8b3aad7df68183fc P 16209ececd2d420588b390f95db9b92a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:16.718338 13935 log.cc:1196] T 23b91b8716e5405e8b3aad7df68183fc P 16209ececd2d420588b390f95db9b92a: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-1/wal/wals/23b91b8716e5405e8b3aad7df68183fc
I20260926 07:36:16.718573 13943 raft_consensus.cc:2781] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:36:16.718969 13935 ts_tablet_manager.cc:1939] T 23b91b8716e5405e8b3aad7df68183fc P 16209ececd2d420588b390f95db9b92a: Deleting consensus metadata
I20260926 07:36:16.721485 13937 tablet_bootstrap.cc:654] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:16.712010 13934 ts_tablet_manager.cc:1918] T 23b91b8716e5405e8b3aad7df68183fc P 095560781b2d4e58b3804fce75312b90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:16.724269 13934 log.cc:1196] T 23b91b8716e5405e8b3aad7df68183fc P 095560781b2d4e58b3804fce75312b90: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/23b91b8716e5405e8b3aad7df68183fc
I20260926 07:36:16.724943 13934 ts_tablet_manager.cc:1939] T 23b91b8716e5405e8b3aad7df68183fc P 095560781b2d4e58b3804fce75312b90: Deleting consensus metadata
I20260926 07:36:16.725873 13936 tablet_bootstrap.cc:492] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90: Bootstrap starting.
I20260926 07:36:16.727306 13947 tablet_bootstrap.cc:492] T 3efbc3ccb32d46919864ad44f7cf630d P a6ff3db459444683972a22263d29009b: No bootstrap required, opened a new log
I20260926 07:36:16.727713 13947 ts_tablet_manager.cc:1397] T 3efbc3ccb32d46919864ad44f7cf630d P a6ff3db459444683972a22263d29009b: Time spent bootstrapping tablet: real 0.062s user 0.005s sys 0.005s
I20260926 07:36:16.727694 13943 raft_consensus.cc:686] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [term 1 LEADER]: Becoming Leader. State: Replica: 095560781b2d4e58b3804fce75312b90, State: Running, Role: LEADER
I20260926 07:36:16.728508 13943 consensus_queue.cc:237] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.729974 13460 heartbeater.cc:503] Master 127.10.171.60:45615 was elected leader, sending a full tablet report...
I20260926 07:36:16.734396 13936 tablet_bootstrap.cc:654] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:16.734442 13934 tablet_replica.cc:330] T 504c993884da427fb5dd210b73dd41af P 095560781b2d4e58b3804fce75312b90: stopping tablet replica
I20260926 07:36:16.736362 13934 raft_consensus.cc:2229] T 504c993884da427fb5dd210b73dd41af P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 07:36:16.742552 13937 tablet_bootstrap.cc:492] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a: No bootstrap required, opened a new log
I20260926 07:36:16.743052 13937 ts_tablet_manager.cc:1397] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a: Time spent bootstrapping tablet: real 0.057s user 0.009s sys 0.000s
I20260926 07:36:16.744961 13947 raft_consensus.cc:348] T 3efbc3ccb32d46919864ad44f7cf630d P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.745946 13934 raft_consensus.cc:2258] T 504c993884da427fb5dd210b73dd41af P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 07:36:16.756068 13947 raft_consensus.cc:374] T 3efbc3ccb32d46919864ad44f7cf630d P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:16.756553 13947 raft_consensus.cc:729] T 3efbc3ccb32d46919864ad44f7cf630d P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ff3db459444683972a22263d29009b, State: Initialized, Role: FOLLOWER
I20260926 07:36:16.759159 13936 tablet_bootstrap.cc:492] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90: No bootstrap required, opened a new log
I20260926 07:36:16.759725 13936 ts_tablet_manager.cc:1397] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90: Time spent bootstrapping tablet: real 0.034s user 0.006s sys 0.006s
I20260926 07:36:16.762324 13936 raft_consensus.cc:348] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:16.762966 13936 raft_consensus.cc:374] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:16.763248 13936 raft_consensus.cc:729] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 095560781b2d4e58b3804fce75312b90, State: Initialized, Role: FOLLOWER
I20260926 07:36:16.763895 13936 consensus_queue.cc:260] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:16.765870 13936 ts_tablet_manager.cc:1428] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260926 07:36:16.766987 13937 raft_consensus.cc:348] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.767699 13937 raft_consensus.cc:374] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:16.767973 13937 raft_consensus.cc:729] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16209ececd2d420588b390f95db9b92a, State: Initialized, Role: FOLLOWER
I20260926 07:36:16.751909 13934 ts_tablet_manager.cc:1905] T 504c993884da427fb5dd210b73dd41af P 095560781b2d4e58b3804fce75312b90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 07:36:16.772522 13947 consensus_queue.cc:260] T 3efbc3ccb32d46919864ad44f7cf630d P a6ff3db459444683972a22263d29009b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.775774 13947 ts_tablet_manager.cc:1428] T 3efbc3ccb32d46919864ad44f7cf630d P a6ff3db459444683972a22263d29009b: Time spent starting tablet: real 0.048s user 0.004s sys 0.004s
I20260926 07:36:16.776698 13947 tablet_bootstrap.cc:492] T 504c993884da427fb5dd210b73dd41af P a6ff3db459444683972a22263d29009b: Bootstrap starting.
I20260926 07:36:16.773792 13937 consensus_queue.cc:260] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.782042 13937 ts_tablet_manager.cc:1428] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a: Time spent starting tablet: real 0.039s user 0.007s sys 0.000s
I20260926 07:36:16.782912 13937 tablet_bootstrap.cc:492] T b310b46797af437ba5d7e5c11195b217 P 16209ececd2d420588b390f95db9b92a: Bootstrap starting.
I20260926 07:36:16.799172 13947 tablet_bootstrap.cc:654] T 504c993884da427fb5dd210b73dd41af P a6ff3db459444683972a22263d29009b: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:16.802484 13937 tablet_bootstrap.cc:654] T b310b46797af437ba5d7e5c11195b217 P 16209ececd2d420588b390f95db9b92a: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:16.809535 13934 ts_tablet_manager.cc:1918] T 504c993884da427fb5dd210b73dd41af P 095560781b2d4e58b3804fce75312b90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 07:36:16.809952 13934 log.cc:1196] T 504c993884da427fb5dd210b73dd41af P 095560781b2d4e58b3804fce75312b90: Deleting WAL directory at /tmp/dist-test-taskNvrOpy/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790408105935953-10924-0/minicluster-data/ts-0/wal/wals/504c993884da427fb5dd210b73dd41af
I20260926 07:36:16.810608 13934 ts_tablet_manager.cc:1939] T 504c993884da427fb5dd210b73dd41af P 095560781b2d4e58b3804fce75312b90: Deleting consensus metadata
I20260926 07:36:16.830662 13937 tablet_bootstrap.cc:492] T b310b46797af437ba5d7e5c11195b217 P 16209ececd2d420588b390f95db9b92a: No bootstrap required, opened a new log
I20260926 07:36:16.831100 13937 ts_tablet_manager.cc:1397] T b310b46797af437ba5d7e5c11195b217 P 16209ececd2d420588b390f95db9b92a: Time spent bootstrapping tablet: real 0.048s user 0.006s sys 0.007s
I20260926 07:36:16.836668 13947 tablet_bootstrap.cc:492] T 504c993884da427fb5dd210b73dd41af P a6ff3db459444683972a22263d29009b: No bootstrap required, opened a new log
I20260926 07:36:16.837067 13947 ts_tablet_manager.cc:1397] T 504c993884da427fb5dd210b73dd41af P a6ff3db459444683972a22263d29009b: Time spent bootstrapping tablet: real 0.061s user 0.013s sys 0.000s
I20260926 07:36:16.837229 13937 raft_consensus.cc:348] T b310b46797af437ba5d7e5c11195b217 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:16.838546 13937 raft_consensus.cc:374] T b310b46797af437ba5d7e5c11195b217 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:16.838864 13937 raft_consensus.cc:729] T b310b46797af437ba5d7e5c11195b217 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16209ececd2d420588b390f95db9b92a, State: Initialized, Role: FOLLOWER
I20260926 07:36:16.839538 13937 consensus_queue.cc:260] T b310b46797af437ba5d7e5c11195b217 P 16209ececd2d420588b390f95db9b92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:16.839733 13947 raft_consensus.cc:348] T 504c993884da427fb5dd210b73dd41af P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.840420 13947 raft_consensus.cc:374] T 504c993884da427fb5dd210b73dd41af P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:16.840691 13947 raft_consensus.cc:729] T 504c993884da427fb5dd210b73dd41af P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ff3db459444683972a22263d29009b, State: Initialized, Role: FOLLOWER
I20260926 07:36:16.841384 13947 consensus_queue.cc:260] T 504c993884da427fb5dd210b73dd41af P a6ff3db459444683972a22263d29009b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.843715 13947 ts_tablet_manager.cc:1428] T 504c993884da427fb5dd210b73dd41af P a6ff3db459444683972a22263d29009b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 07:36:16.844540 13947 tablet_bootstrap.cc:492] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b: Bootstrap starting.
I20260926 07:36:16.847139 13937 ts_tablet_manager.cc:1428] T b310b46797af437ba5d7e5c11195b217 P 16209ececd2d420588b390f95db9b92a: Time spent starting tablet: real 0.016s user 0.003s sys 0.003s
I20260926 07:36:16.850711 13947 tablet_bootstrap.cc:654] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b: Neither blocks nor log segments found. Creating new log.
I20260926 07:36:16.855654 13947 tablet_bootstrap.cc:492] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b: No bootstrap required, opened a new log
I20260926 07:36:16.856171 13947 ts_tablet_manager.cc:1397] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20260926 07:36:16.858570 13947 raft_consensus.cc:348] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.859386 13947 raft_consensus.cc:374] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 07:36:16.859711 13947 raft_consensus.cc:729] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ff3db459444683972a22263d29009b, State: Initialized, Role: FOLLOWER
I20260926 07:36:16.860493 13947 consensus_queue.cc:260] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.862766 13947 ts_tablet_manager.cc:1428] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b: Time spent starting tablet: real 0.006s user 0.000s sys 0.004s
I20260926 07:36:16.908315 13944 raft_consensus.cc:482] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:16.908752 13944 raft_consensus.cc:504] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.910529 13944 leader_election.cc:288] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
I20260926 07:36:16.912180 13413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "504c993884da427fb5dd210b73dd41af" candidate_uuid: "16209ececd2d420588b390f95db9b92a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ff3db459444683972a22263d29009b" is_pre_election: true
I20260926 07:36:16.912794 13413 raft_consensus.cc:2454] T 504c993884da427fb5dd210b73dd41af P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16209ececd2d420588b390f95db9b92a in term 0.
W20260926 07:36:16.912883 13193 leader_election.cc:341] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af
I20260926 07:36:16.913784 13193 leader_election.cc:302] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16209ececd2d420588b390f95db9b92a, a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90
I20260926 07:36:16.914817 13944 raft_consensus.cc:2781] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:36:16.915119 13944 raft_consensus.cc:482] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:36:16.915407 13944 raft_consensus.cc:3037] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:36:16.921625 13944 raft_consensus.cc:504] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
W20260926 07:36:16.932742 13193 leader_election.cc:341] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af
I20260926 07:36:16.934857 13413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "504c993884da427fb5dd210b73dd41af" candidate_uuid: "16209ececd2d420588b390f95db9b92a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ff3db459444683972a22263d29009b"
I20260926 07:36:16.935398 13413 raft_consensus.cc:3037] T 504c993884da427fb5dd210b73dd41af P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:36:16.941866 13413 raft_consensus.cc:2454] T 504c993884da427fb5dd210b73dd41af P a6ff3db459444683972a22263d29009b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16209ececd2d420588b390f95db9b92a in term 1.
I20260926 07:36:16.943027 13193 leader_election.cc:302] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16209ececd2d420588b390f95db9b92a, a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90
I20260926 07:36:16.947953 13944 leader_election.cc:288] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 election: Requested vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
I20260926 07:36:16.949003 13944 raft_consensus.cc:2781] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:36:16.949970 13944 raft_consensus.cc:686] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a [term 1 LEADER]: Becoming Leader. State: Replica: 16209ececd2d420588b390f95db9b92a, State: Running, Role: LEADER
I20260926 07:36:16.951216 13944 consensus_queue.cc:237] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:16.988672 13943 raft_consensus.cc:482] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:16.989166 13943 raft_consensus.cc:504] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:16.991169 13943 leader_election.cc:288] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
I20260926 07:36:16.993230 13264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b310b46797af437ba5d7e5c11195b217" candidate_uuid: "095560781b2d4e58b3804fce75312b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16209ececd2d420588b390f95db9b92a" is_pre_election: true
I20260926 07:36:16.993824 13264 raft_consensus.cc:2454] T b310b46797af437ba5d7e5c11195b217 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 095560781b2d4e58b3804fce75312b90 in term 0.
W20260926 07:36:16.993992 13034 leader_election.cc:341] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: b310b46797af437ba5d7e5c11195b217
I20260926 07:36:16.994951 13035 leader_election.cc:302] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a; no voters: a6ff3db459444683972a22263d29009b
I20260926 07:36:16.995744 13943 raft_consensus.cc:2781] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 07:36:16.996101 13943 raft_consensus.cc:482] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 07:36:16.996578 13943 raft_consensus.cc:3037] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:36:17.002738 13943 raft_consensus.cc:504] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:17.004560 13943 leader_election.cc:288] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 [CANDIDATE]: Term 1 election: Requested vote from peers 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
I20260926 07:36:17.005340 13264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b310b46797af437ba5d7e5c11195b217" candidate_uuid: "095560781b2d4e58b3804fce75312b90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16209ececd2d420588b390f95db9b92a"
I20260926 07:36:17.005889 13264 raft_consensus.cc:3037] T b310b46797af437ba5d7e5c11195b217 P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Advancing to term 1
W20260926 07:36:17.006780 13034 leader_election.cc:341] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: b310b46797af437ba5d7e5c11195b217
I20260926 07:36:17.011909 13264 raft_consensus.cc:2454] T b310b46797af437ba5d7e5c11195b217 P 16209ececd2d420588b390f95db9b92a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 095560781b2d4e58b3804fce75312b90 in term 1.
I20260926 07:36:17.013163 13035 leader_election.cc:302] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a; no voters: a6ff3db459444683972a22263d29009b
I20260926 07:36:17.013929 13943 raft_consensus.cc:2781] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 07:36:17.014381 13943 raft_consensus.cc:686] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 [term 1 LEADER]: Becoming Leader. State: Replica: 095560781b2d4e58b3804fce75312b90, State: Running, Role: LEADER
I20260926 07:36:17.017059 13943 consensus_queue.cc:237] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
W20260926 07:36:17.032984 13036 connection.cc:535] server connection from 127.0.0.1:51090 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:36:17.033025 13348 connection.cc:535] server connection from 127.0.0.1:42408 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:36:17.033125 13194 connection.cc:535] server connection from 127.0.0.1:51490 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:36:17.035351 13472 connection.cc:535] client connection to 127.10.171.60:45615 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:36:17.036314 13035 connection.cc:535] client connection to 127.10.171.60:45615 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:36:17.037545 13472 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.10.171.60:45615)
W20260926 07:36:17.038358 13472 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.171.60:45615)
W20260926 07:36:17.039047 13472 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.10.171.60:45615)
W20260926 07:36:17.040053 13348 connection.cc:535] client connection to 127.10.171.60:45615 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 07:36:17.044032 13472 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.10.171.60:45615)
W20260926 07:36:17.049707 13455 debug-util.cc:398] Leaking SignalData structure 0x7b08000e48c0 after lost signal to thread 13328
I20260926 07:36:17.190249 13974 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:36:17.190881 13974 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:17.201823 13974 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:36:17.202363 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:36:17.206790 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:36:17.207799 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:36:17.208263 13839 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:17.209074 13974 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:36:17.582551 13978 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:17.584787 13978 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
W20260926 07:36:17.589089 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:36:17.592278 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:36:17.592694 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:36:17.599050 13978 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
I20260926 07:36:17.600062 13978 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:36:17.634384 13979 raft_consensus.cc:482] T 3efbc3ccb32d46919864ad44f7cf630d P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:17.634868 13979 raft_consensus.cc:504] T 3efbc3ccb32d46919864ad44f7cf630d P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:17.637756 13108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3efbc3ccb32d46919864ad44f7cf630d" candidate_uuid: "a6ff3db459444683972a22263d29009b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "095560781b2d4e58b3804fce75312b90" is_pre_election: true
I20260926 07:36:17.638012 13264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3efbc3ccb32d46919864ad44f7cf630d" candidate_uuid: "a6ff3db459444683972a22263d29009b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16209ececd2d420588b390f95db9b92a" is_pre_election: true
I20260926 07:36:17.638777 13264 raft_consensus.cc:2379] T 3efbc3ccb32d46919864ad44f7cf630d P 16209ececd2d420588b390f95db9b92a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a6ff3db459444683972a22263d29009b in current term 1: Already voted for candidate 095560781b2d4e58b3804fce75312b90 in this term.
I20260926 07:36:17.639468 13979 leader_election.cc:288] T 3efbc3ccb32d46919864ad44f7cf630d P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
I20260926 07:36:17.640769 13348 leader_election.cc:302] T 3efbc3ccb32d46919864ad44f7cf630d P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:36:17.641570 13979 raft_consensus.cc:3037] T 3efbc3ccb32d46919864ad44f7cf630d P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Advancing to term 1
I20260926 07:36:17.648691 13979 raft_consensus.cc:2726] T 3efbc3ccb32d46919864ad44f7cf630d P a6ff3db459444683972a22263d29009b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:36:17.767714 13980 consensus_queue.cc:1035] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 07:36:17.776978 13983 consensus_queue.cc:1035] T 3efbc3ccb32d46919864ad44f7cf630d P 095560781b2d4e58b3804fce75312b90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260926 07:36:17.785497 13034 consensus_peers.cc:487] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 -> Peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Couldn't send request to peer a6ff3db459444683972a22263d29009b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b310b46797af437ba5d7e5c11195b217. This is attempt 1: this message will repeat every 5th retry.
I20260926 07:36:17.799054 13985 consensus_queue.cc:1035] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 07:36:17.887246 13987 consensus_queue.cc:1035] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 07:36:17.934752 10924 master-stress-test.cc:578] Killing test thread 1/10
I20260926 07:36:17.960462 13995 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:36:17.960959 13995 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
W20260926 07:36:17.975970 13994 log.cc:924] Time spent T 504c993884da427fb5dd210b73dd41af P a6ff3db459444683972a22263d29009b: Append to log took a long time: real 0.062s user 0.001s sys 0.000s
W20260926 07:36:18.036571 13840 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.171.62:36693, user_credentials={real_user=slave}} blocked reactor thread for 62086.2us
I20260926 07:36:18.039929 13995 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:36:18.043320 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 1: this message will repeat every 5th retry.
W20260926 07:36:18.049049 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:36:18.051079 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:36:18.052290 13840 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:18.059589 13995 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:36:18.636597 14009 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:36:18.637125 14009 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:18.639348 14009 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:36:18.655925 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:36:18.657122 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:36:18.657549 13839 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:18.658308 14009 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:36:18.720745 14015 ts_tablet_manager.cc:927] T b310b46797af437ba5d7e5c11195b217 P a6ff3db459444683972a22263d29009b: Initiating tablet copy from peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253)
I20260926 07:36:18.725129 14015 tablet_copy_client.cc:316] T b310b46797af437ba5d7e5c11195b217 P a6ff3db459444683972a22263d29009b: tablet copy: Beginning tablet copy session from remote peer at address 127.10.171.1:37253
I20260926 07:36:18.727087 13128 tablet_copy_service.cc:140] P 095560781b2d4e58b3804fce75312b90: Received BeginTabletCopySession request for tablet b310b46797af437ba5d7e5c11195b217 from peer a6ff3db459444683972a22263d29009b ({username='slave'} at 127.10.171.3:47151)
I20260926 07:36:18.727610 13128 tablet_copy_service.cc:161] P 095560781b2d4e58b3804fce75312b90: Beginning new tablet copy session on tablet b310b46797af437ba5d7e5c11195b217 from peer a6ff3db459444683972a22263d29009b at {username='slave'} at 127.10.171.3:47151: session id = a6ff3db459444683972a22263d29009b-b310b46797af437ba5d7e5c11195b217
I20260926 07:36:18.735240 13128 tablet_copy_source_session.cc:205] T b310b46797af437ba5d7e5c11195b217 P 095560781b2d4e58b3804fce75312b90: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 07:36:18.738857 14015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b310b46797af437ba5d7e5c11195b217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 07:36:18.750339 14015 tablet_copy_client.cc:791] T b310b46797af437ba5d7e5c11195b217 P a6ff3db459444683972a22263d29009b: tablet copy: Starting download of 0 data blocks...
I20260926 07:36:18.751048 14015 tablet_copy_client.cc:655] T b310b46797af437ba5d7e5c11195b217 P a6ff3db459444683972a22263d29009b: tablet copy: Starting download of 1 WAL segments...
I20260926 07:36:18.754655 14015 tablet_copy_client.cc:531] T b310b46797af437ba5d7e5c11195b217 P a6ff3db459444683972a22263d29009b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 07:36:18.760454 14015 tablet_bootstrap.cc:492] T b310b46797af437ba5d7e5c11195b217 P a6ff3db459444683972a22263d29009b: Bootstrap starting.
I20260926 07:36:18.776834 14015 tablet_bootstrap.cc:492] T b310b46797af437ba5d7e5c11195b217 P a6ff3db459444683972a22263d29009b: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 07:36:18.777755 14015 tablet_bootstrap.cc:492] T b310b46797af437ba5d7e5c11195b217 P a6ff3db459444683972a22263d29009b: Bootstrap complete.
I20260926 07:36:18.778378 14015 ts_tablet_manager.cc:1397] T b310b46797af437ba5d7e5c11195b217 P a6ff3db459444683972a22263d29009b: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.005s
I20260926 07:36:18.781140 14015 raft_consensus.cc:348] T b310b46797af437ba5d7e5c11195b217 P a6ff3db459444683972a22263d29009b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:18.781908 14015 raft_consensus.cc:729] T b310b46797af437ba5d7e5c11195b217 P a6ff3db459444683972a22263d29009b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ff3db459444683972a22263d29009b, State: Initialized, Role: FOLLOWER
I20260926 07:36:18.782660 14015 consensus_queue.cc:260] T b310b46797af437ba5d7e5c11195b217 P a6ff3db459444683972a22263d29009b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:18.785840 14015 ts_tablet_manager.cc:1428] T b310b46797af437ba5d7e5c11195b217 P a6ff3db459444683972a22263d29009b: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260926 07:36:18.793835 13128 tablet_copy_service.cc:342] P 095560781b2d4e58b3804fce75312b90: Request end of tablet copy session a6ff3db459444683972a22263d29009b-b310b46797af437ba5d7e5c11195b217 received from {username='slave'} at 127.10.171.3:47151
I20260926 07:36:18.794231 13128 tablet_copy_service.cc:433] P 095560781b2d4e58b3804fce75312b90: ending tablet copy session a6ff3db459444683972a22263d29009b-b310b46797af437ba5d7e5c11195b217 on tablet b310b46797af437ba5d7e5c11195b217 with peer a6ff3db459444683972a22263d29009b
I20260926 07:36:19.612161 13414 raft_consensus.cc:1203] T b310b46797af437ba5d7e5c11195b217 P a6ff3db459444683972a22263d29009b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1] Dedup: 1.1->[]
I20260926 07:36:19.777551 14034 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:36:19.777981 14034 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:19.779412 14034 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:36:19.783676 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:36:19.784086 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:36:19.784476 13839 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:19.785037 14034 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:36:20.909229 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:36:20.909576 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:36:21.390373 14053 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:21.390930 14053 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:21.392788 14053 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:36:21.394606 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:36:21.395103 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:36:21.395495 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:36:21.396358 14053 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:36:21.461083 14054 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:36:21.461480 14054 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:21.462857 14054 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:36:21.467126 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:36:21.467629 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:36:21.467957 13839 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:21.468567 14054 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:36:22.798548 14067 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:22.798952 14067 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:22.800503 14067 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:36:22.801900 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:36:22.802348 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:36:22.802587 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:36:22.803067 14067 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:36:24.037354 14083 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:36:24.037750 14083 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:24.039134 14083 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:36:24.041100 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260926 07:36:24.043665 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:36:24.044081 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:36:24.044461 13840 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:24.044994 14083 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:36:25.912585 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:36:25.913079 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:36:27.810161 14148 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:36:27.810626 14148 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:27.812089 14148 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:36:27.816689 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:36:27.817293 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:36:27.817575 13839 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:27.818280 14148 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:36:27.894522 14152 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:27.895114 14152 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:27.897439 14152 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:36:27.898988 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:36:27.899609 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:36:27.899873 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:36:27.900475 14152 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:36:28.235862 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 6: this message will repeat every 5th retry.
I20260926 07:36:28.642793 14159 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:28.643313 14159 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:28.645457 14159 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:36:28.646853 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:36:28.647473 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:36:28.647771 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:36:28.648365 14159 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:36:30.915560 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:36:30.915922 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:36:31.562660 14205 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:36:31.563066 14205 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:31.564559 14205 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:36:31.566375 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 07:36:31.568781 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:36:31.569162 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:36:31.569492 13840 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:31.570060 14205 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:36:33.743498 14241 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:33.743883 14241 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:33.745467 14241 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:36:33.746905 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:36:33.747298 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:36:33.747581 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:36:33.748098 14241 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:36:35.477286 13455 debug-util.cc:398] Leaking SignalData structure 0x7b08000c3b60 after lost signal to thread 13328
W20260926 07:36:35.478752 13455 debug-util.cc:398] Leaking SignalData structure 0x7b080006f3a0 after lost signal to thread 13458
I20260926 07:36:35.918993 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:36:35.919459 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:36:37.443341 14282 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:37.443876 14282 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:37.445744 14282 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:36:37.447304 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:36:37.447887 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:36:37.448226 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:36:37.448805 14282 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:36:39.394196 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 11: this message will repeat every 5th retry.
I20260926 07:36:40.048767 14309 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:40.049336 14309 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:40.051349 14309 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:36:40.053086 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:36:40.053666 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:36:40.053927 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:36:40.054466 14309 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:36:40.921907 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:36:40.922286 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:36:42.817710 14345 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:36:42.818150 14345 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:42.819775 14345 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:36:42.821676 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 07:36:42.824359 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:36:42.824803 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:36:42.825119 13840 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:42.825703 14345 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260926 07:36:44.095358 13455 debug-util.cc:398] Leaking SignalData structure 0x7b08000d4d40 after lost signal to thread 13328
W20260926 07:36:44.096422 13455 debug-util.cc:398] Leaking SignalData structure 0x7b08000e21a0 after lost signal to thread 13458
I20260926 07:36:44.950404 14376 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:44.950791 14376 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:44.952689 14376 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:36:44.954172 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:36:44.954648 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:36:44.954962 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:36:44.955453 14376 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:36:45.693384 14387 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:36:45.693918 14387 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:45.695465 14387 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:36:45.700546 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:36:45.700963 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:36:45.701275 13840 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:45.701817 14387 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:36:45.924444 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:36:45.924825 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:36:48.067606 14427 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:48.068102 14427 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:36:48.070499 14427 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:36:48.071938 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:36:48.072535 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:36:48.072840 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:36:48.073534 14427 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:36:49.972635 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 16: this message will repeat every 5th retry.
I20260926 07:36:50.322703 14459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:36:50.323132 14459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:50.324806 14459 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:36:50.326606 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 07:36:50.329067 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:36:50.329504 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:36:50.329823 13840 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:50.330278 14459 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:36:50.927320 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:36:50.927634 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:36:55.139635 14526 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:36:55.140413 14526 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:36:55.142799 14526 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:36:55.144620 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:36:55.145215 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:36:55.145565 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:36:55.146178 14526 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:36:55.930753 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:36:55.931311 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:36:56.992985 14560 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:36:56.993460 14560 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:36:56.994964 14560 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:36:56.997043 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 07:36:56.999542 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:36:56.999915 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:36:57.000285 13840 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:36:57.000869 14560 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260926 07:36:57.461642 13149 debug-util.cc:398] Leaking SignalData structure 0x7b08000e7720 after lost signal to thread 13010
W20260926 07:36:57.462519 13149 debug-util.cc:398] Leaking SignalData structure 0x7b08000e27a0 after lost signal to thread 13152
W20260926 07:37:00.679253 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 21: this message will repeat every 5th retry.
I20260926 07:37:00.933807 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:37:00.934171 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:37:01.492362 14625 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:37:01.492774 14625 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:37:01.494135 14625 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:37:01.498313 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:37:01.498734 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:37:01.499014 13839 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:37:01.499559 14625 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:37:02.684363 14643 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:37:02.684909 14643 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:37:02.686866 14643 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:37:02.688675 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:37:02.689254 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:37:02.689576 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:37:02.690182 14643 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:37:04.107144 14669 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:37:04.107542 14669 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:37:04.109661 14669 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:37:04.111219 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:37:04.111644 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:37:04.112011 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:37:04.112650 14669 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:37:05.937479 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:37:05.937950 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:37:07.376060 13311 heartbeater.cc:644] Failed to heartbeat to 127.10.171.62:36693 (156 consecutive failures): Network error: Failed to ping master at 127.10.171.62:36693: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:37:07.394590 13154 heartbeater.cc:644] Failed to heartbeat to 127.10.171.60:45615 (106 consecutive failures): Network error: Failed to ping master at 127.10.171.60:45615: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:37:07.458385 13460 heartbeater.cc:644] Failed to heartbeat to 127.10.171.60:45615 (106 consecutive failures): Network error: Failed to ping master at 127.10.171.60:45615: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:37:07.897145 14727 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:37:07.897900 14727 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:37:07.899999 14727 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:37:07.901376 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:37:07.901834 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:37:07.902133 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:37:07.902702 14727 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:37:10.430400 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 26: this message will repeat every 5th retry.
I20260926 07:37:10.940178 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:37:10.940557 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:37:14.970058 14841 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:37:14.970681 14841 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:37:14.973073 14841 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:37:14.974514 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:37:14.974996 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:37:14.975235 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:37:14.975756 14841 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:37:15.944017 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:37:15.944615 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:37:20.456197 14924 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:37:20.456708 14924 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:37:20.458680 14924 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:37:20.460547 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 07:37:20.462950 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:37:20.463352 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:37:20.463634 13840 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:37:20.464102 14924 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260926 07:37:20.664342 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 31: this message will repeat every 5th retry.
I20260926 07:37:20.946491 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:37:20.946832 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:37:22.980504 14953 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:37:22.981316 14953 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:37:22.984038 14953 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:37:22.986848 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:37:22.989249 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:37:22.989794 13347 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:37:22.990545 14953 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:37:25.950165 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:37:25.950645 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:37:29.842206 15055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:37:29.842715 15055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:37:29.844374 15055 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:37:29.846504 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 07:37:29.849114 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:37:29.849578 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:37:29.849915 13840 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:37:29.850540 15055 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:37:30.953291 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:37:30.953667 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:37:30.991554 15071 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:37:30.992141 15071 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:37:30.994304 15071 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:37:30.996094 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:37:30.996662 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:37:30.996940 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:37:30.997460 15071 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:37:31.388825 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 36: this message will repeat every 5th retry.
I20260926 07:37:34.517258 15096 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:37:34.518028 15096 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:37:34.520776 15096 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:37:34.522291 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:37:34.522775 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:37:34.523021 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:37:34.523602 15096 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:37:35.957297 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:37:35.957741 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:37:38.325250 15124 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:37:38.325616 15124 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:37:38.326973 15124 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:37:38.328433 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:37:38.328850 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:37:38.329054 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:37:38.329473 15124 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:37:40.432571 13884 debug-util.cc:398] Leaking SignalData structure 0x7b08000501e0 after lost signal to thread 13806
W20260926 07:37:40.433137 13884 debug-util.cc:398] Leaking SignalData structure 0x7b08000522a0 after lost signal to thread 13887
W20260926 07:37:40.913053 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 41: this message will repeat every 5th retry.
I20260926 07:37:40.979315 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:37:40.979656 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:37:42.118942 13305 debug-util.cc:398] Leaking SignalData structure 0x7b0800043ea0 after lost signal to thread 13172
W20260926 07:37:42.119869 13305 debug-util.cc:398] Leaking SignalData structure 0x7b08000af0c0 after lost signal to thread 13308
I20260926 07:37:42.313436 15161 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:37:42.314049 15161 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:37:42.316339 15161 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:37:42.318168 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:37:42.318840 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:37:42.319240 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:37:42.319888 15161 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:37:42.639230 13149 debug-util.cc:398] Leaking SignalData structure 0x7b08000e2660 after lost signal to thread 13010
W20260926 07:37:42.640594 13149 debug-util.cc:398] Leaking SignalData structure 0x7b08000dc680 after lost signal to thread 13152
I20260926 07:37:45.982607 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:37:45.983027 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:37:48.935729 13149 debug-util.cc:398] Leaking SignalData structure 0x7b08000d7ea0 after lost signal to thread 13010
W20260926 07:37:48.936570 13149 debug-util.cc:398] Leaking SignalData structure 0x7b08000e99a0 after lost signal to thread 13152
I20260926 07:37:49.789099 15247 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:37:49.789741 15247 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:37:49.792022 15247 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:37:49.793960 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 07:37:49.796772 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:37:49.797242 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:37:49.797528 13840 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:37:49.798053 15247 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260926 07:37:50.977778 13455 debug-util.cc:398] Leaking SignalData structure 0x7b08000d3680 after lost signal to thread 13328
W20260926 07:37:50.978613 13455 debug-util.cc:398] Leaking SignalData structure 0x7b08000da640 after lost signal to thread 13458
I20260926 07:37:50.986172 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:37:50.986608 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:37:51.044904 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 46: this message will repeat every 5th retry.
I20260926 07:37:55.989836 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:37:55.990257 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:37:56.571424 15352 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:37:56.571861 15352 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:37:56.573560 15352 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:37:56.575882 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 07:37:56.580205 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:37:56.580783 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:37:56.581185 13839 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:37:56.581919 15352 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:37:57.111910 15365 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:37:57.112493 15365 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:37:57.114670 15365 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:37:57.116245 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:37:57.116739 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:37:57.117116 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:37:57.117686 15365 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:37:58.367322 15386 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:37:58.368027 15386 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:37:58.370276 15386 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:37:58.371733 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:37:58.372247 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:37:58.372519 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:37:58.373080 15386 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:38:00.992537 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:38:00.992892 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:38:01.342458 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 51: this message will repeat every 5th retry.
I20260926 07:38:02.379390 15439 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:38:02.379868 15439 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:38:02.381479 15439 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:38:02.383924 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 07:38:02.386220 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:38:02.386574 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:38:02.386864 13840 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:38:02.387338 15439 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:38:04.206682 15457 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:38:04.207140 15457 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:38:04.208823 15457 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:38:04.212976 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:38:04.213402 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:38:04.213678 13839 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:38:04.214105 15457 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:38:05.995599 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:38:05.996032 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:38:07.559800 13311 heartbeater.cc:644] Failed to heartbeat to 127.10.171.62:36693 (308 consecutive failures): Network error: Failed to ping master at 127.10.171.62:36693: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:38:07.785789 13154 heartbeater.cc:644] Failed to heartbeat to 127.10.171.60:45615 (226 consecutive failures): Network error: Failed to ping master at 127.10.171.60:45615: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:38:07.790652 13460 heartbeater.cc:644] Failed to heartbeat to 127.10.171.60:45615 (226 consecutive failures): Network error: Failed to ping master at 127.10.171.60:45615: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:38:10.998427 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:38:10.998780 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:38:11.323331 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 56: this message will repeat every 5th retry.
I20260926 07:38:15.121176 15589 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:38:15.121740 15589 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:38:15.123863 15589 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:38:15.125593 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:38:15.126016 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:38:15.126224 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:38:15.126689 15589 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:38:15.881028 13842 maintenance_manager.cc:419] P 7726b3d3f73544318efe4a3be7a63f70: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033345
I20260926 07:38:15.950232 13841 maintenance_manager.cc:643] P 7726b3d3f73544318efe4a3be7a63f70: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.068s user 0.037s sys 0.014s Metrics: {"bytes_written":12447,"cfile_init":1,"drs_written":1,"lbm_read_time_us":200,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2143,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"rows_written":22,"thread_start_us":437,"threads_started":1,"wal-append.queue_time_us":769}
I20260926 07:38:15.952724 13842 maintenance_manager.cc:419] P 7726b3d3f73544318efe4a3be7a63f70: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1247 bytes on disk
I20260926 07:38:15.954675 13841 maintenance_manager.cc:643] P 7726b3d3f73544318efe4a3be7a63f70: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":306,"lbm_reads_lt_1ms":4}
I20260926 07:38:16.001878 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:38:16.002278 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:38:17.486618 15622 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:38:17.487218 15622 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:38:17.489538 15622 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:38:17.491075 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:38:17.491614 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:38:17.491881 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:38:17.492434 15622 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:38:19.318439 13149 debug-util.cc:398] Leaking SignalData structure 0x7b08000c2da0 after lost signal to thread 13010
W20260926 07:38:19.319231 13149 debug-util.cc:398] Leaking SignalData structure 0x7b08000c8680 after lost signal to thread 13152
I20260926 07:38:19.359267 15653 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:38:19.359745 15653 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:38:19.361644 15653 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:38:19.363509 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 07:38:19.366483 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:38:19.366911 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:38:19.367197 13840 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:38:19.367656 15653 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:38:21.004717 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:38:21.005059 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:38:21.038929 15673 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:38:21.039366 15673 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:38:21.041330 15673 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:38:21.058761 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:38:21.059185 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:38:21.059541 13839 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:38:21.060168 15673 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260926 07:38:21.288348 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 61: this message will repeat every 5th retry.
I20260926 07:38:25.222113 15748 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:38:25.222723 15748 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:38:25.224502 15748 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:38:25.226022 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:38:25.226558 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:38:25.226774 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:38:25.227234 15748 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:38:26.008158 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:38:26.008595 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:38:27.381089 15788 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:38:27.381613 15788 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:38:27.383332 15788 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:38:27.385715 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:38:27.386790 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:38:27.387156 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:38:27.387799 15788 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:38:27.861802 13305 debug-util.cc:398] Leaking SignalData structure 0x7b08000c2960 after lost signal to thread 13172
W20260926 07:38:27.862573 13305 debug-util.cc:398] Leaking SignalData structure 0x7b08000c62e0 after lost signal to thread 13308
W20260926 07:38:30.891459 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 66: this message will repeat every 5th retry.
I20260926 07:38:31.011096 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:38:31.011397 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:38:36.014358 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:38:36.014750 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:38:38.615465 15989 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:38:38.616147 15989 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:38:38.618053 15989 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:38:38.619632 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:38:38.620215 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:38:38.620486 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:38:38.621034 15989 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:38:40.424937 16029 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:38:40.425427 16029 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:38:40.427534 16029 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:38:40.429482 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 07:38:40.432163 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:38:40.432581 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:38:40.432917 13840 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:38:40.433449 16029 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:38:40.880589 16041 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:38:40.881161 16041 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:38:40.883255 16041 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:38:40.885176 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:38:40.885655 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:38:40.885952 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:38:40.886469 16041 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:38:41.017033 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:38:41.017370 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:38:41.210516 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 71: this message will repeat every 5th retry.
I20260926 07:38:46.042030 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:38:46.042619 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:38:46.199445 13884 debug-util.cc:398] Leaking SignalData structure 0x7b08000bf640 after lost signal to thread 13806
W20260926 07:38:46.199936 13884 debug-util.cc:398] Leaking SignalData structure 0x7b08000bbe60 after lost signal to thread 13887
W20260926 07:38:50.897786 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 76: this message will repeat every 5th retry.
I20260926 07:38:51.045310 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:38:51.045717 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:38:51.361333 16227 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:38:51.361842 16227 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:38:51.363401 16227 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:38:51.365028 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:38:51.365497 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:38:51.365758 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:38:51.366309 16227 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:38:52.760787 16258 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:38:52.761245 16258 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:38:52.762693 16258 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:38:52.764978 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 07:38:52.767395 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:38:52.767836 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:38:52.768180 13840 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:38:52.768704 16258 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:38:56.049171 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:38:56.049729 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:38:56.431442 16318 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:38:56.432044 16318 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:38:56.434228 16318 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:38:56.435881 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:38:56.436363 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:38:56.436628 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:38:56.437127 16318 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:38:58.219786 16346 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:38:58.220440 16346 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:38:58.223323 16346 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:38:58.224958 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:38:58.225519 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:38:58.225818 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:38:58.226408 16346 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:38:59.710250 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 81: this message will repeat every 5th retry.
I20260926 07:39:01.052508 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:39:01.052896 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:39:05.482016 16432 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:39:05.482538 16432 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:39:05.484311 16432 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:39:05.486478 13839 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 07:39:05.489351 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:39:05.489799 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:39:05.490083 13839 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:39:05.490545 16432 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:39:06.055459 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:39:06.055837 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:39:06.256695 13455 debug-util.cc:398] Leaking SignalData structure 0x7b0800085a00 after lost signal to thread 13328
W20260926 07:39:06.257813 13455 debug-util.cc:398] Leaking SignalData structure 0x7b0800041f20 after lost signal to thread 13458
W20260926 07:39:07.996507 13311 heartbeater.cc:644] Failed to heartbeat to 127.10.171.62:36693 (457 consecutive failures): Network error: Failed to ping master at 127.10.171.62:36693: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:39:08.115907 13461 heartbeater.cc:644] Failed to heartbeat to 127.10.171.62:36693 (375 consecutive failures): Network error: Failed to ping master at 127.10.171.62:36693: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:39:08.118278 13154 heartbeater.cc:644] Failed to heartbeat to 127.10.171.60:45615 (346 consecutive failures): Network error: Failed to ping master at 127.10.171.60:45615: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:39:10.152592 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 86: this message will repeat every 5th retry.
I20260926 07:39:11.058197 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:39:11.058581 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:39:12.155342 16526 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:12.155866 16526 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:39:12.157815 16526 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:39:12.159425 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:39:12.159868 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:39:12.160162 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:39:12.160609 16526 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:39:13.982772 16553 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:13.983440 16553 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:39:13.986034 16553 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:39:13.987723 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:39:13.988276 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:39:13.988512 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:39:13.989050 16553 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:39:15.508634 13149 debug-util.cc:398] Leaking SignalData structure 0x7b0800042ec0 after lost signal to thread 13010
W20260926 07:39:15.509517 13149 debug-util.cc:398] Leaking SignalData structure 0x7b08000e6f60 after lost signal to thread 13152
I20260926 07:39:16.061837 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:39:16.062296 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:39:20.104450 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 91: this message will repeat every 5th retry.
I20260926 07:39:20.539533 16665 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:39:20.540060 16665 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:39:20.542599 16665 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:39:20.544677 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 07:39:20.547875 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:39:20.548502 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:39:20.548769 13839 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:39:20.549367 16665 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:39:20.693086 16671 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:20.693850 16671 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:39:20.695703 16671 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:39:20.697923 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:39:20.698454 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:39:20.698840 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:39:20.699599 16671 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:39:21.064839 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:39:21.065212 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:39:21.967662 16690 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:21.968035 16690 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:39:21.969686 16690 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:39:21.971060 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:39:21.971501 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:39:21.971792 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:39:21.972355 16690 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:39:26.068660 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:39:26.069139 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:39:28.500669 16805 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:28.501235 16805 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:39:28.503415 16805 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:39:28.504994 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:39:28.505537 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:39:28.505805 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:39:28.506371 16805 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:39:30.042935 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 96: this message will repeat every 5th retry.
I20260926 07:39:31.071599 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:39:31.071928 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:39:31.475715 16847 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:31.476253 16847 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:39:31.478523 16847 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:39:31.480022 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:39:31.480455 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:39:31.480655 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:39:31.481134 16847 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:39:34.250478 16900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:39:34.250939 16900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:39:34.252653 16900 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:39:34.255754 13839 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 07:39:34.259822 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:39:34.260332 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:39:34.260597 13840 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:39:34.261068 16900 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:39:35.058917 16912 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:35.059469 16912 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:39:35.061194 16912 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:39:35.062566 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:39:35.063023 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:39:35.063285 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:39:35.063795 16912 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:39:36.075001 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:39:36.075430 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:39:39.811870 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 101: this message will repeat every 5th retry.
I20260926 07:39:41.077932 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:39:41.078269 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:39:46.081151 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:39:46.081583 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:39:47.299901 17123 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:47.300807 17123 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:39:47.303620 17123 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:39:47.305406 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:39:47.305902 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:39:47.306149 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:39:47.306651 17123 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:39:49.363708 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 106: this message will repeat every 5th retry.
I20260926 07:39:51.084474 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:39:51.084945 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:39:51.477567 17204 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:51.477991 17204 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:39:51.479784 17204 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:39:51.481163 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:39:51.481581 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:39:51.481782 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:39:51.482254 17204 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:39:52.988106 17230 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:39:52.988644 17230 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:39:52.990470 17230 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:39:52.992637 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 07:39:52.995541 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:39:52.996589 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:39:52.996915 13840 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:39:52.997570 17230 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:39:54.744820 17261 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:39:54.745347 17261 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:39:54.748258 17261 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:39:54.749260 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:39:54.755435 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:39:54.755913 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:39:54.756762 17261 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:39:56.088104 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:39:56.088671 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:39:58.436909 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 111: this message will repeat every 5th retry.
I20260926 07:40:01.091094 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:40:01.091408 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:40:02.809581 13305 debug-util.cc:398] Leaking SignalData structure 0x7b08000441a0 after lost signal to thread 13172
W20260926 07:40:02.810411 13305 debug-util.cc:398] Leaking SignalData structure 0x7b08000c13c0 after lost signal to thread 13308
I20260926 07:40:04.540684 17415 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:40:04.541255 17415 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:40:04.543615 17415 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:40:04.545450 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:40:04.545921 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:40:04.546263 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:40:04.546876 17415 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:40:05.419224 17426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:40:05.419734 17426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:40:05.421469 17426 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:40:05.423799 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 07:40:05.426666 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:40:05.427098 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:40:05.427416 13840 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:40:05.428001 17426 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:40:06.093979 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:40:06.094343 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:40:06.487694 17449 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:40:06.488232 17449 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:40:06.490180 17449 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:40:06.491700 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:40:06.492185 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:40:06.492404 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:40:06.492847 17449 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:40:07.585567 17468 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:40:07.586153 17468 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:40:07.587908 17468 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:40:07.589280 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:40:07.589764 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:40:07.590044 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:40:07.590517 17468 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:40:08.167685 13310 heartbeater.cc:644] Failed to heartbeat to 127.10.171.60:45615 (576 consecutive failures): Network error: Failed to ping master at 127.10.171.60:45615: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:40:08.464561 13460 heartbeater.cc:644] Failed to heartbeat to 127.10.171.60:45615 (466 consecutive failures): Network error: Failed to ping master at 127.10.171.60:45615: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:40:08.480836 13154 heartbeater.cc:644] Failed to heartbeat to 127.10.171.60:45615 (466 consecutive failures): Network error: Failed to ping master at 127.10.171.60:45615: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:40:08.896592 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 116: this message will repeat every 5th retry.
I20260926 07:40:11.097249 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:40:11.097625 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:40:12.965224 13884 debug-util.cc:398] Leaking SignalData structure 0x7b0800098ca0 after lost signal to thread 13806
W20260926 07:40:12.965679 13884 debug-util.cc:398] Leaking SignalData structure 0x7b080008bc20 after lost signal to thread 13887
I20260926 07:40:16.110944 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:40:16.111356 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:40:17.971552 17645 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:40:17.972229 17645 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:40:17.974133 17645 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:40:17.975771 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:40:17.976408 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:40:17.976725 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:40:17.977298 17645 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:40:18.407958 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 121: this message will repeat every 5th retry.
I20260926 07:40:21.113965 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:40:21.114328 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:40:22.253209 17710 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:40:22.253636 17710 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:40:22.255434 17710 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:40:22.257200 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:40:22.257700 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:40:22.258116 13347 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:40:22.258666 17710 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:40:22.569427 17711 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:40:22.569931 17711 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:40:22.571950 17711 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:40:22.573643 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:40:22.574167 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:40:22.574493 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:40:22.574975 17711 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:40:24.464426 17746 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:40:24.464924 17746 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:40:24.466763 17746 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:40:24.468633 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 07:40:24.471464 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:40:24.471958 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:40:24.472335 13840 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:40:24.472908 17746 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:40:26.117383 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:40:26.117825 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:40:26.403684 13149 debug-util.cc:398] Leaking SignalData structure 0x7b08000d2180 after lost signal to thread 13010
W20260926 07:40:26.404595 13149 debug-util.cc:398] Leaking SignalData structure 0x7b08000b97e0 after lost signal to thread 13152
W20260926 07:40:27.991737 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 126: this message will repeat every 5th retry.
I20260926 07:40:31.120579 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:40:31.120934 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:40:31.412611 17868 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:40:31.413154 17868 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:40:31.415274 17868 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:40:31.416924 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:40:31.417483 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:40:31.417796 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:40:31.418411 17868 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:40:32.737767 17888 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:40:32.738308 17888 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:40:32.739907 17888 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:40:32.742156 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 07:40:32.744589 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:40:32.745059 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:40:32.745322 13840 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:40:32.745805 17888 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:40:36.124612 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:40:36.125043 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:40:38.067142 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 131: this message will repeat every 5th retry.
I20260926 07:40:39.926656 18008 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:40:39.927166 18008 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:40:39.929091 18008 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:40:39.930625 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:40:39.931206 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:40:39.931445 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:40:39.932050 18008 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:40:41.127980 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:40:41.128456 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:40:43.240331 18066 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:40:43.240707 18066 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:40:43.242203 18066 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:40:43.243752 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:40:43.244174 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:40:43.244467 13347 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:40:43.244983 18066 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:40:46.131399 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:40:46.131857 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:40:47.717921 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 136: this message will repeat every 5th retry.
I20260926 07:40:48.226290 18153 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:40:48.226760 18153 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:40:48.229034 18153 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:40:48.230320 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:40:48.230927 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:40:48.231202 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:40:48.231777 18153 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:40:48.604766 18167 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:40:48.605185 18167 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:40:48.607005 18167 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:40:48.608892 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 07:40:48.612102 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:40:48.612602 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:40:48.612876 13840 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:40:48.613444 18167 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:40:51.134356 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:40:51.134712 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:40:53.808609 13455 debug-util.cc:398] Leaking SignalData structure 0x7b08000c18c0 after lost signal to thread 13328
W20260926 07:40:53.809595 13455 debug-util.cc:398] Leaking SignalData structure 0x7b08000e81a0 after lost signal to thread 13458
I20260926 07:40:56.137820 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:40:56.138312 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 07:40:58.089444 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 141: this message will repeat every 5th retry.
I20260926 07:40:58.831879 18337 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:40:58.832810 18337 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:40:58.836189 18337 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:40:58.838351 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:40:58.838891 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:40:58.839244 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:40:58.839900 18337 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:41:01.140712 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:41:01.141052 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:41:01.964721 18384 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:41:01.965221 18384 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:41:01.966974 18384 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:41:01.968313 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:41:01.968838 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:41:01.969097 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
I20260926 07:41:01.969623 18384 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:41:02.022193 18390 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:41:02.022588 18390 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:41:02.024003 18390 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:41:02.025254 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:41:02.025640 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:41:02.025959 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:41:02.026438 18390 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:41:06.144029 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:41:06.144526 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:41:06.794365 18466 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:41:06.794790 18466 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:41:06.796878 18466 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:41:06.798910 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260926 07:41:06.803576 18465 raft_consensus.cc:482] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:41:06.805145 18465 raft_consensus.cc:504] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } } peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } }
I20260926 07:41:06.812203 18465 leader_election.cc:288] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), a6ff3db459444683972a22263d29009b (127.10.171.3:34663)
W20260926 07:41:06.813364 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ff3db459444683972a22263d29009b (127.10.171.3:34663): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
W20260926 07:41:06.822714 13193 leader_election.cc:341] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 385ef5d71f3a49c8b5057525a388230d
I20260926 07:41:06.823168 13193 leader_election.cc:302] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16209ececd2d420588b390f95db9b92a; no voters: 095560781b2d4e58b3804fce75312b90, a6ff3db459444683972a22263d29009b
W20260926 07:41:06.824325 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:41:06.827111 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:41:06.827517 13839 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:41:06.828248 18466 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:41:06.830479 18465 raft_consensus.cc:2726] T 385ef5d71f3a49c8b5057525a388230d P 16209ececd2d420588b390f95db9b92a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 07:41:07.000228 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 146: this message will repeat every 5th retry.
W20260926 07:41:08.222335 13310 heartbeater.cc:644] Failed to heartbeat to 127.10.171.60:45615 (727 consecutive failures): Network error: Failed to ping master at 127.10.171.60:45615: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:41:08.816701 13158 heartbeater.cc:644] Failed to heartbeat to 127.10.171.62:36693 (615 consecutive failures): Network error: Failed to ping master at 127.10.171.62:36693: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:41:08.858798 13461 heartbeater.cc:644] Failed to heartbeat to 127.10.171.62:36693 (615 consecutive failures): Network error: Failed to ping master at 127.10.171.62:36693: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
I20260926 07:41:09.441920 18503 raft_consensus.cc:482] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 07:41:09.442442 18503 raft_consensus.cc:504] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ff3db459444683972a22263d29009b" member_type: VOTER last_known_addr { host: "127.10.171.3" port: 34663 } } peers { permanent_uuid: "095560781b2d4e58b3804fce75312b90" member_type: VOTER last_known_addr { host: "127.10.171.1" port: 37253 } } peers { permanent_uuid: "16209ececd2d420588b390f95db9b92a" member_type: VOTER last_known_addr { host: "127.10.171.2" port: 39565 } }
I20260926 07:41:09.444096 18503 leader_election.cc:288] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253), 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565)
W20260926 07:41:09.445526 13347 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
W20260926 07:41:09.446044 13348 leader_election.cc:341] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16209ececd2d420588b390f95db9b92a (127.10.171.2:39565): Not found: Tablet not found: 23b91b8716e5405e8b3aad7df68183fc
I20260926 07:41:09.446275 13348 leader_election.cc:302] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ff3db459444683972a22263d29009b; no voters: 095560781b2d4e58b3804fce75312b90, 16209ececd2d420588b390f95db9b92a
I20260926 07:41:09.446702 18503 raft_consensus.cc:2726] T 23b91b8716e5405e8b3aad7df68183fc P a6ff3db459444683972a22263d29009b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 07:41:11.146886 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:41:11.147284 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:41:13.097297 18554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:41:13.097697 18554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:41:13.099114 18554 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:41:13.100946 13840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 07:41:13.103878 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:41:13.104532 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:41:13.104794 13839 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:41:13.105233 18554 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 07:41:13.626922 18567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a6ef379bc314e798568d59eba3a3a91)
I20260926 07:41:13.627295 18567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51342b641d47457d84f48fcbb6c48375" member_type: VOTER last_known_addr { host: "127.10.171.62" port: 36693 } } peers { permanent_uuid: "7726b3d3f73544318efe4a3be7a63f70" member_type: VOTER last_known_addr { host: "127.10.171.61" port: 45551 } } peers { permanent_uuid: "0a6ef379bc314e798568d59eba3a3a91" member_type: VOTER last_known_addr { host: "127.10.171.60" port: 45615 } }
I20260926 07:41:13.628800 18567 leader_election.cc:288] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693), 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615)
W20260926 07:41:13.633131 13840 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 51342b641d47457d84f48fcbb6c48375 (127.10.171.62:36693): Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111)
W20260926 07:41:13.633513 13839 leader_election.cc:334] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0a6ef379bc314e798568d59eba3a3a91 (127.10.171.60:45615): Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
I20260926 07:41:13.633812 13839 leader_election.cc:302] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7726b3d3f73544318efe4a3be7a63f70; no voters: 0a6ef379bc314e798568d59eba3a3a91, 51342b641d47457d84f48fcbb6c48375
I20260926 07:41:13.634316 18567 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7726b3d3f73544318efe4a3be7a63f70 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260926 07:41:15.153743 13305 debug-util.cc:398] Leaking SignalData structure 0x7b08000da5e0 after lost signal to thread 13172
W20260926 07:41:15.154639 13305 debug-util.cc:398] Leaking SignalData structure 0x7b08000de840 after lost signal to thread 13308
I20260926 07:41:16.150647 13801 master_runner.cc:186] Error getting master registration for 127.10.171.62:36693: Network error: Client connection negotiation failed: client connection to 127.10.171.62:36693: connect: Connection refused (error 111),
I20260926 07:41:16.151084 13801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 07:41:16.859302 13842 maintenance_manager.cc:419] P 7726b3d3f73544318efe4a3be7a63f70: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1247 bytes on disk
I20260926 07:41:16.870065 13841 maintenance_manager.cc:643] P 7726b3d3f73544318efe4a3be7a63f70: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.010s user 0.002s sys 0.008s Metrics: {"dirs.queue_time_us":723,"dirs.run_cpu_time_us":218,"dirs.run_wall_time_us":218,"thread_start_us":710,"threads_started":2,"wal-append.queue_time_us":659}
W20260926 07:41:17.299012 13193 consensus_peers.cc:487] T 504c993884da427fb5dd210b73dd41af P 16209ececd2d420588b390f95db9b92a -> Peer 095560781b2d4e58b3804fce75312b90 (127.10.171.1:37253): Couldn't send request to peer 095560781b2d4e58b3804fce75312b90. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 504c993884da427fb5dd210b73dd41af. This is attempt 151: this message will repeat every 5th retry.
W20260926 07:41:17.317001 13473 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:41:17.317479 13473 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:41:17.317687 13473 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
W20260926 07:41:17.317873 13473 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
F20260926 07:41:17.329402 13480 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5af6599ea22546fdb4c598d6a124f28d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
F20260926 07:41:17.332235 13481 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_daf8f6d0f2c6493d879b9a3bfbd1cfad on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.10.171.62:36693,127.10.171.61:45551,127.10.171.60:45615, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.10.171.60:45615: connect: Connection refused (error 111)
*** Check failure stack trace: ***
*** Aborted at 1790408477 (unix time) try "date -d @1790408477" if you are using GNU date ***
*** Check failure stack trace: ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800002aac) received by PID 10924 (TID 0x7f24500fc700) from PID 10924; stack trace: ***
@ 0x2b0af0 __tsan::CallUserSignalHandler() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
@ 0x2b31e0 rtl_sigaction() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
@ 0x7f2461444980 (unknown) at ??:0
@ 0x7f245c701e87 gsignal at ??:0
@ 0x7f245c7037f1 abort at ??:0
@ 0x2b4f37 __interceptor_abort at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
@ 0x7f245f735df4 google::LogMessage::SendToLog() at ??:0
@ 0x7f245f7369a0 google::LogMessage::Flush() at ??:0
@ 0x7f245f73badb google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x3572c4 kudu::MasterStressTest::CreateTableThread() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/master-stress-test.cc:?
@ 0x3874ce _ZNSt3__18__invokeIMN4kudu16MasterStressTestEFvvEPNS1_26MasterStressTest_Test_TestEJEvEEDTcldsdeclsr3std3__1E7forwardIT0_Efp0_Efp_spclsr3std3__1E7forwardIT1_Efp1_EEEOT_OS7_DpOS8_ at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/type_traits:3840
@ 0x387381 _ZNSt3__116__thread_executeINS_10unique_ptrINS_15__thread_structENS_14default_deleteIS2_EEEEMN4kudu16MasterStressTestEFvvEJPNS6_26MasterStressTest_Test_TestEEJLm2EEEEvRNS_5tupleIJT_T0_DpT1_EEENS_15__tuple_indicesIJXspT2_EEEE at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/thread:281
@ 0x386acb std::__1::__thread_proxy<>() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/thread:?
@ 0x2b11b0 __tsan_thread_start_func at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:959
@ 0x7f24614396db start_thread at ??:0
@ 0x7f245c7e461f clone at ??:0